You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@geode.apache.org by Dharam Thacker <dh...@gmail.com> on 2017/01/01 06:40:54 UTC
Re: JSON in Apache geode
Hi John,
Thanks for the example! My query is more related to schemaless entity.
*Actually this example refers to domain class "Customer" and converts pojo
to JSON. But that also means it has "schema" [Well defined domain model --
We know number of attributes and type of attributes in advance].*
I am looking for JSON but I don't have "schema" defined as "Customer". Why?
Because if I setup schema/domain class, then I have to frequently change my
domain class and database while onboarding new column or any upgrades to
existing column,
Thus above step will force me to clear & destroy existing region and
recreate it with new model class and pdx metadata.
*Let's say from "Source" itself I am receiving json as string and then I
want to store such json document as pdx serializable in region. *
Then I have 3 ways which I am replicating once again from my first email as
below! Any thoughts for that?
*Case1: Primary key <java.lang.String>*
region.put(String.valueOf(i), JSONFormatter.fromJSON(obj.toJSONString()));
For above, I believe it should be --> Region<String,PdxInstance> then how
does it look like in spring data geode xml configuration?
*Case2: Primary Key <? extends java.lang.Serializable>*
region.put (new CustomObject() , JSONFormatter.fromJSON(obj.
toJSONString()));
For above, I believe it should be --> Region<CustomObject,PdxInstance> then
how does it look like in spring data geode xml configuration?
*Case3: Is below case valid one and recommended ?*
@Region
Class JSONRegion {
CustomPrimaryKeyObject primaryKey;
JSONObject jsonObject; [org.simple.Json library or Jackson library]
// Constructor + Getter/Setter
}
Thanks & Regards,
- Dharam Thacker
On Sun, Jan 1, 2017 at 4:53 AM, John Blum <jb...@pivotal.io> wrote:
> Also note, here is an example...
>
> https://github.com/spring-projects/spring-gemfire-examples/tree/master/
> quickstart/json
>
> On Sat, Dec 31, 2016 at 3:22 PM, John Blum <jb...@pivotal.io> wrote:
>
>> Dharam-
>>
>> You should take a look at this...
>>
>> http://docs.spring.io/spring-data-gemfire/docs/current/refer
>> ence/html/#bootstrap:region:json
>>
>> Also, be aware that if you stick a (JSON) "String" in a Region as a value
>> (i.e. Region.put(key, stringValue)) then the type of the value will be a
>> String, not a PDX instance as I assume you would expect. Geode has no idea
>> whether you are sticking JSON into a Region or a String.
>>
>> Additionally, using JSONFormatter is clunky and limiting since you are
>> now tied to the JSON processing implementation of JSONFormatter.
>> Likewise, it only knows about JSON Strings (PdxInstance:fromJSON(:String))
>> and PDX Instances (toJSON(:PdxInstance)). What happens if you pass in
>> an Object? What then?
>>
>> Under-the-hood, the SDG Region auto-proxy support uses JSONFormatter,
>> but also knows what to do with a POJO if a Region.put(key, Object)
>> occurs since that Region is proxied to store JSON, err... PDX content.
>>
>> You must have the JSON data stored in Geode as a PdxInstance to be able
>> to index and query the JSON document.
>>
>> Hope this helps!
>>
>> -John
>>
>>
>>
>>
>> On Sat, Dec 31, 2016 at 8:59 AM, Dharam Thacker <
>> dharamthacker88@gmail.com> wrote:
>>
>>> Hi Team,
>>>
>>> Wish you all a very happy new year!
>>>
>>> Is there any example on json processing with apache geode?
>>>
>>> Could you help us with below examples on how region definition would
>>> look like?
>>>
>>> *Example*:
>>>
>>> *Case1: Primary key <java.lang.String>*
>>>
>>> region.put(String.valueOf(i), JSONFormatter.fromJSON(obj.toJSONString()));
>>>
>>> For above, I believe it should be --> Region<String,PdxInstance> then
>>> how does it look like in spring data geode xml configuration?
>>>
>>>
>>> *Case2: Primary Key <? extends java.lang.Serializable>*
>>>
>>> region.put (new CustomObject() , JSONFormatter.fromJSON(obj.toJ
>>> SONString()));
>>>
>>> For above, I believe it should be --> Region<CustomObject,PdxInstance>
>>> then how does it look like in spring data geode xml configuration?
>>>
>>> *Case3: Is below case valid one and recommended ?*
>>>
>>> @Region
>>> Class JSONRegion {
>>> CustomPrimaryKeyObject primaryKey;
>>> JSONObject jsonObject;
>>>
>>> // Constructor + Getter/Setter
>>> }
>>>
>>>
>>> More is there any limitation/pros-cons if we store JSON document as
>>> value vs schema bound domain java object?
>>>
>>> Thanks & Regards,
>>> - Dharam Thacker
>>>
>>
>>
>>
>> --
>> -John
>> john.blum10101 (skype)
>>
>
>
>
> --
> -John
> john.blum10101 (skype)
>
Re: JSON in Apache geode
Posted by Hitesh Khamesra <hi...@yahoo.com>.
Just to add here, you can add "JSONFormatter.fromJSON(obj.toJ SONString()" for region keys as well, but need to set pdx "read-serialized="true" for that. This will consider all the fields of JSON for hashcode/equals.
From: Dharam Thacker <dh...@gmail.com>
To: user@geode.apache.org
Sent: Sunday, January 1, 2017 11:13 AM
Subject: Re: JSON in Apache geode
Hi Udo,
Yeah but I have constraint from database side as well which follows some schema in terms of fields. If any field request comes then I will have to add new field to database + matching domain object model and then populate it back to cache
[Above process usually requires code changes and production turn over and increases time to market for such frequent changes]
I will have to clear the cache still I believe! Let's say I have 100 Instrument for Equity with 10 fields & in future I have been requested to add new column which must be present for all 100 instruments already in memory!
In that case, I will have to clear existing instruments and reload new instrument data with 11 fields. I may defer not to destroy existing region even though I have pdx metdadata persistent to disk store other than region persistent disk store. Isn't it ?
Regards,
Dharam
- Dharam Thacker
On Sun, Jan 1, 2017 at 10:51 PM, Udo Kohlmeyer <ud...@apache.org> wrote:
Hi there Dharam, The power of PDX is that you can have object versioning. This means, that you can add/remove fields to an existing type without the fear of having to clear & destroy the region. PDX will help you with multiple versions of the same Customer object. If you were to access a v1 of the customer the fields from the newer versions would just be null. In addition if a client were to access a v3 Customer object and it only had the v1 Customer object definition, it would only be able access the fields that the v1 object knew about. It will not lose the v3 field information. You could test this out by starting a server, and then have a client push data with different customer object versions.
This WAS part of the reason to go from DataSerializable (no versions) to PDXSerializable (versions). The ONLY thing you'll have to watch out for is to change the TYPE of a field definition. You can add and remove fields at hearts content, but don't change the type of an existing field. --Udo
On 12/31/16 22:40, Dharam Thacker wrote:
Hi John,
Thanks for the example! My query is more related to schemaless entity.
Actually this example refers to domain class "Customer" and converts pojo to JSON. But that also means it has "schema" [Well defined domain model -- We know number of attributes and type of attributes in advance].
I am looking for JSON but I don't have "schema" defined as "Customer". Why? Because if I setup schema/domain class, then I have to frequently change my domain class and database while onboarding new column or any upgrades to existing column,
Thus above step will force me to clear & destroy existing region and recreate it with new model class and pdx metadata.
Let's say from "Source" itself I am receiving json as string and then I want to store such json document as pdx serializable in region.
Then I have 3 ways which I am replicating once again from my first email as below! Any thoughts for that?
Case1: Primary key <java.lang.String>
region.put(String.valueOf(i), JSONFormatter.fromJSON(obj.toJ SONString()));For above, I believe it should be --> Region<String,PdxInstance> then how does it look like in spring data geode xml configuration?Case2: Primary Key <? extends java.lang.Serializable>region.put (new CustomObject() , JSONFormatter.fromJSON(obj.toJ SONString()));For above, I believe it should be --> Region<CustomObject,PdxInstanc e> then how does it look like in spring data geode xml configuration?Case3: Is below case valid one and recommended ?@RegionClass JSONRegion { CustomPrimaryKeyObject primaryKey; JSONObject jsonObject; [org.simple.Json library or Jackson library] // Constructor + Getter/Setter}Thanks & Regards,- Dharam ThackerOn Sun, Jan 1, 2017 at 4:53 AM, John Blum <jb...@pivotal.io> wrote:
Also note, here is an example...https://github.com/spring-proj ects/spring-gemfire-examples/ tree/master/quickstart/jsonOn Sat, Dec 31, 2016 at 3:22 PM, John Blum <jb...@pivotal.io> wrote:
Dharam-You should take a look at this...http://docs.spring.io/spring-d ata-gemfire/docs/current/refer ence/html/#bootstrap:region:js onAlso, be aware that if you stick a (JSON) "String" in a Region as a value (i.e. Region.put(key, stringValue)) then the type of the value will be a String, not a PDX instance as I assume you would expect. Geode has no idea whether you are sticking JSON into a Region or a String.Additionally, using JSONFormatter is clunky and limiting since you are now tied to the JSON processing implementation of JSONFormatter. Likewise, it only knows about JSON Strings (PdxInstance:fromJSON(:String) ) and PDX Instances (toJSON(:PdxInstance)). What happens if you pass in an Object? What then?Under-the-hood, the SDG Region auto-proxy support uses JSONFormatter, but also knows what to do with a POJO if a Region.put(key, Object) occurs since that Region is proxied to store JSON, err... PDX content.You must have the JSON data stored in Geode as a PdxInstance to be able to index and query the JSON document.Hope this helps!-JohnOn Sat, Dec 31, 2016 at 8:59 AM, Dharam Thacker <dh...@gmail.com> wrote:
Hi Team,Wish you all a very happy new year!Is there any example on json processing with apache geode? Could you help us with below examples on how region definition would look like?Example:Case1: Primary key <java.lang.String>
region.put(String.valueOf(i), JSONFormatter.fromJSON(obj.toJ SONString()));For above, I believe it should be --> Region<String,PdxInstance> then how does it look like in spring data geode xml configuration?Case2: Primary Key <? extends java.lang.Serializable>region.put (new CustomObject() , JSONFormatter.fromJSON(obj.toJ SONString()));For above, I believe it should be --> Region<CustomObject,PdxInstanc e> then how does it look like in spring data geode xml configuration?Case3: Is below case valid one and recommended ?@RegionClass JSONRegion { CustomPrimaryKeyObject primaryKey; JSONObject jsonObject; // Constructor + Getter/Setter}More is there any limitation/pros-cons if we store JSON document as value vs schema bound domain java object?Thanks & Regards,- Dharam Thacker
-- -Johnjohn.blum10101 (skype)
-- -Johnjohn.blum10101 (skype)
Re: JSON in Apache geode
Posted by Dharam Thacker <dh...@gmail.com>.
Hi Udo,
Yeah but I have constraint from database side as well which follows some
schema in terms of fields. If any field request comes then I will have to
add new field to database + matching domain object model and then populate
it back to cache
[Above process usually requires code changes and production turn over and
increases time to market for such frequent changes]
*I will have to clear the cache still I believe! Let's say I have 100
Instrument for Equity with 10 fields & in future I have been requested to
add new column which must be present for all 100 instruments already in
memory!*
*In that case, I will have to clear existing instruments and reload new
instrument data with 11 fields. I may defer not to destroy existing region
even though I have pdx metdadata persistent to disk store other than region
persistent disk store. Isn't it ?*
Regards,
Dharam
- Dharam Thacker
On Sun, Jan 1, 2017 at 10:51 PM, Udo Kohlmeyer <ud...@apache.org> wrote:
> Hi there Dharam,
>
> The power of PDX is that you can have object versioning. This means, that
> you can add/remove fields to an existing type without the fear of having to
> clear & destroy the region. PDX will help you with multiple versions of the
> same Customer object. If you were to access a v1 of the customer the fields
> from the newer versions would just be null. In addition if a client were to
> access a v3 Customer object and it only had the v1 Customer object
> definition, it would only be able access the fields that the v1 object knew
> about. It will not lose the v3 field information.
>
> You could test this out by starting a server, and then have a client push
> data with different customer object versions.
>
> This WAS part of the reason to go from DataSerializable (no versions) to
> PDXSerializable (versions).
>
> The ONLY thing you'll have to watch out for is to change the TYPE of a
> field definition. You can add and remove fields at hearts content, but
> don't change the type of an existing field.
>
> --Udo
>
> On 12/31/16 22:40, Dharam Thacker wrote:
>
> Hi John,
>
> Thanks for the example! My query is more related to schemaless entity.
>
> *Actually this example refers to domain class "Customer" and converts pojo
> to JSON. But that also means it has "schema" [Well defined domain model --
> We know number of attributes and type of attributes in advance].*
>
> I am looking for JSON but I don't have "schema" defined as "Customer".
> Why? Because if I setup schema/domain class, then I have to frequently
> change my domain class and database while onboarding new column or any
> upgrades to existing column,
>
> Thus above step will force me to clear & destroy existing region and
> recreate it with new model class and pdx metadata.
>
> *Let's say from "Source" itself I am receiving json as string and then I
> want to store such json document as pdx serializable in region. *
>
> Then I have 3 ways which I am replicating once again from my first email
> as below! Any thoughts for that?
>
> *Case1: Primary key <java.lang.String>*
>
> region.put(String.valueOf(i), JSONFormatter.fromJSON(obj.toJSONString()));
>
> For above, I believe it should be --> Region<String,PdxInstance> then how
> does it look like in spring data geode xml configuration?
> *Case2: Primary Key <? extends java.lang.Serializable>*
> region.put (new CustomObject() , JSONFormatter.fromJSON(obj.toJSONString()));
>
> For above, I believe it should be --> Region<CustomObject,PdxInstance>
> then how does it look like in spring data geode xml configuration?
> *Case3: Is below case valid one and recommended ?*
> @Region
> Class JSONRegion {
> CustomPrimaryKeyObject primaryKey;
> JSONObject jsonObject; [org.simple.Json library or Jackson library]
> // Constructor + Getter/Setter
> }
> Thanks & Regards,
> - Dharam Thacker
> On Sun, Jan 1, 2017 at 4:53 AM, John Blum <jb...@pivotal.io> wrote:
>>
>> Also note, here is an example...
>> https://github.com/spring-projects/spring-gemfire-examples/
>> tree/master/quickstart/json
>> On Sat, Dec 31, 2016 at 3:22 PM, John Blum <jb...@pivotal.io> wrote:
>>>
>>> Dharam-
>>> You should take a look at this...
>>> http://docs.spring.io/spring-data-gemfire/docs/current/refer
>>> ence/html/#bootstrap:region:json
>>> Also, be aware that if you stick a (JSON) "String" in a Region as a
>>> value (i.e. Region.put(key, stringValue)) then the type of the value
>>> will be a String, not a PDX instance as I assume you would expect. Geode
>>> has no idea whether you are sticking JSON into a Region or a String.
>>> Additionally, using JSONFormatter is clunky and limiting since you are
>>> now tied to the JSON processing implementation of JSONFormatter.
>>> Likewise, it only knows about JSON Strings (
>>> PdxInstance:fromJSON(:String)) and PDX Instances (toJSON(:PdxInstance)).
>>> What happens if you pass in an Object? What then?
>>> Under-the-hood, the SDG Region auto-proxy support uses JSONFormatter,
>>> but also knows what to do with a POJO if a Region.put(key, Object)
>>> occurs since that Region is proxied to store JSON, err... PDX content.
>>> You must have the JSON data stored in Geode as a PdxInstance to be able
>>> to index and query the JSON document.
>>> Hope this helps!
>>> -John
>>> On Sat, Dec 31, 2016 at 8:59 AM, Dharam Thacker <
>>> dharamthacker88@gmail.com> wrote:
>>>>
>>>> Hi Team,
>>>> Wish you all a very happy new year!
>>>> Is there any example on json processing with apache geode? Could you
>>>> help us with below examples on how region definition would look like?
>>>> *Example*:
>>>>
>>>> *Case1: Primary key <java.lang.String>*
>>>>
>>>> region.put(String.valueOf(i), JSONFormatter.fromJSON(obj.toJSONString()));
>>>>
>>>> For above, I believe it should be --> Region<String,PdxInstance> then
>>>> how does it look like in spring data geode xml configuration?
>>>> *Case2: Primary Key <? extends java.lang.Serializable>*
>>>> region.put (new CustomObject() , JSONFormatter.fromJSON(obj.toJSONString()));
>>>>
>>>> For above, I believe it should be --> Region<CustomObject,PdxInstance>
>>>> then how does it look like in spring data geode xml configuration?
>>>> *Case3: Is below case valid one and recommended ?*
>>>> @Region
>>>> Class JSONRegion {
>>>> CustomPrimaryKeyObject primaryKey;
>>>> JSONObject jsonObject;
>>>> // Constructor + Getter/Setter
>>>> }
>>>> More is there any limitation/pros-cons if we store JSON document as
>>>> value vs schema bound domain java object?
>>>> Thanks & Regards,
>>>> - Dharam Thacker
>>>>
>>> --
>>> -John
>>> john.blum10101 (skype)
>>>
>> --
>> -John
>> john.blum10101 (skype)
>>
>
Re: JSON in Apache geode
Posted by Udo Kohlmeyer <ud...@apache.org>.
Hi there Dharam,
The power of PDX is that you can have object versioning. This means,
that you can add/remove fields to an existing type without the fear of
having to clear & destroy the region. PDX will help you with multiple
versions of the same Customer object. If you were to access a v1 of the
customer the fields from the newer versions would just be null. In
addition if a client were to access a v3 Customer object and it only had
the v1 Customer object definition, it would only be able access the
fields that the v1 object knew about. It will not lose the v3 field
information.
You could test this out by starting a server, and then have a client
push data with different customer object versions.
This WAS part of the reason to go from DataSerializable (no versions) to
PDXSerializable (versions).
The ONLY thing you'll have to watch out for is to change the TYPE of a
field definition. You can add and remove fields at hearts content, but
don't change the type of an existing field.
--Udo
On 12/31/16 22:40, Dharam Thacker wrote:
> Hi John,
>
> Thanks for the example! My query is more related to schemaless entity.
>
> *Actually this example refers to domain class "Customer" and converts
> pojo to JSON. But that also means it has "schema" [Well defined domain
> model -- We know number of attributes and type of attributes in advance].*
>
> I am looking for JSON but I don't have "schema" defined as "Customer".
> Why? Because if I setup schema/domain class, then I have to frequently
> change my domain class and database while onboarding new column or any
> upgrades to existing column,
>
> Thus above step will force me to clear & destroy existing region and
> recreate it with new model class and pdx metadata.
>
> _*Let's say from "Source" itself I am receiving json as string and
> then I want to store such json document as pdx serializable in region. *_
>
> Then I have 3 ways which I am replicating once again from my first
> email as below! Any thoughts for that?
>
> _*Case1: Primary key <java.lang.String>*_
>
> region.put(String.valueOf(i), JSONFormatter.fromJSON(obj.toJSONString()));
> For above, I believe it should be --> Region<String,PdxInstance> then
> how does it look like in spring data geode xml configuration?
> _*Case2: Primary Key <? extends java.lang.Serializable>*_
> region.put (new CustomObject() ,
> JSONFormatter.fromJSON(obj.toJSONString()));
> For above, I believe it should be --> Region<CustomObject,PdxInstance>
> then how does it look like in spring data geode xml configuration?
> _*Case3: Is below case valid one and recommended ?*_
> @Region
> Class JSONRegion {
> CustomPrimaryKeyObject primaryKey;
> JSONObject jsonObject; [org.simple.Json library or Jackson library]
> // Constructor + Getter/Setter
> }
> Thanks & Regards,
> - Dharam Thacker
> On Sun, Jan 1, 2017 at 4:53 AM, John Blum <jblum@pivotal.io
> <ma...@pivotal.io>> wrote:
>
> Also note, here is an example...
> https://github.com/spring-projects/spring-gemfire-examples/tree/master/quickstart/json
> <https://github.com/spring-projects/spring-gemfire-examples/tree/master/quickstart/json>
>
> On Sat, Dec 31, 2016 at 3:22 PM, John Blum <jblum@pivotal.io
> <ma...@pivotal.io>> wrote:
>
> Dharam-
> You should take a look at this...
> http://docs.spring.io/spring-data-gemfire/docs/current/reference/html/#bootstrap:region:json
> <http://docs.spring.io/spring-data-gemfire/docs/current/reference/html/#bootstrap:region:json>
>
> Also, be aware that if you stick a (JSON) "String" in a Region
> as a value (i.e. Region.put(key, stringValue)) then the type
> of the value will be a String, not a PDX instance as I assume
> you would expect. Geode has no idea whether you are sticking
> JSON into a Region or a String.
> Additionally, using JSONFormatter is clunky and limiting since
> you are now tied to the JSON processing implementation of
> JSONFormatter. Likewise, it only knows about JSON Strings
> (PdxInstance:fromJSON(:String)) and PDX Instances
> (toJSON(:PdxInstance)). What happens if you pass in an
> Object? What then?
> Under-the-hood, the SDG Region auto-proxy support uses
> JSONFormatter, but also knows what to do with a POJO if a
> Region.put(key, Object) occurs since that Region is proxied to
> store JSON, err... PDX content.
> You must have the JSON data stored in Geode as a PdxInstance
> to be able to index and query the JSON document.
> Hope this helps!
> -John
> On Sat, Dec 31, 2016 at 8:59 AM, Dharam Thacker
> <dharamthacker88@gmail.com <ma...@gmail.com>>
> wrote:
>
> Hi Team,
> Wish you all a very happy new year!
> Is there any example on json processing with apache geode?
> Could you help us with below examples on how region
> definition would look like?
> _*Example*_:
>
> _*Case1: Primary key <java.lang.String>*_
>
> region.put(String.valueOf(i), JSONFormatter.fromJSON(obj.toJSONString()));
>
> For above, I believe it should be -->
> Region<String,PdxInstance> then how does it look like in
> spring data geode xml configuration?
> _*Case2: Primary Key <? extends java.lang.Serializable>*_
> region.put (new CustomObject() ,
> JSONFormatter.fromJSON(obj.toJSONString()));
> For above, I believe it should be -->
> Region<CustomObject,PdxInstance> then how does it look
> like in spring data geode xml configuration?
> _*Case3: Is below case valid one and recommended ?*_
> @Region
> Class JSONRegion {
> CustomPrimaryKeyObject primaryKey;
> JSONObject jsonObject;
> // Constructor + Getter/Setter
> }
> More is there any limitation/pros-cons if we store JSON
> document as value vs schema bound domain java object?
> Thanks & Regards,
> - Dharam Thacker
>
> --
> -John
> john.blum10101 (skype)
>
> --
> -John
> john.blum10101 (skype)
>