You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@gora.apache.org by Lewis John Mcgibbney <le...@gmail.com> on 2016/03/24 04:45:44 UTC

Modeling Complex Object Values in Gora

Hi Folks,
I'm close to finishing off my long overdue patch for adding Gora support to
Log4J 2.X [0] however I have a question which I hope I've explained well
enough.
In my implementation, the 'object' I am modeling is a Gora instance of the
Log4j 2.X NoSqlObject [1].
As you can see this object is relatively simple with numerous set(key,
value) methods where keys are always strings.
The data modeling issue I am having is how to model the complex value if it
is, for example, an Object, Object[ ], NoSqlObject or NoSqlObject[ ]?
I _think_ I encountered a similar issue a long time aho and turned to Avro
IDL to sort the issue out however before I do that I was thinking something
along the lines of the following

{
  "name": "GoraNoSqlObject",
  "type": "record",
  "namespace": "org.apache.logging.log4j.nosql.appender.gora",
  "doc": "GoraNoSqlObject represents the Gora implementation of NoSqlObject.",
  "fields": [
    {
      "name": "keySimpleValue",
      "type": {
        "type": "map",
        "values": ["null","NoSqlObject"]
      },
      "doc": "Sets the value of a property on this object to a nested
complex object.",
      "default": {
        ...
      }
    },
...

 This however would assume that I could model 'NoSqlObject' elsewhere, and
that it had been compiled and available on the GoraCompiler classpath prior
to this GoraNoSqlObject class being compiled. To me this just doesn't look
right.
I just want to get feedback from anyone interested in this to see how they
would build the data model.
Thanks
Lewis

[0] https://issues.apache.org/jira/browse/LOG4J2-603
[1]
https://logging.apache.org/log4j/2.0/log4j-nosql/apidocs/index.html?org/apache/logging/log4j/nosql/appender/NoSqlObject.html

-- 
*Lewis*

Re: Modeling Complex Object Values in Gora

Posted by Lewis John Mcgibbney <le...@gmail.com>.
I think that 2 is probably how this is done down in Avro however I am not
100% sure mate.
If you look at the Gora Jira space and query the issues by component
gora-compiler you can see that the compiler needs a bit of TLC. I'll prob
try to start in this issue towards the end of the week. Input on the
relevant JIRA issue would be grand if folk want to help out.
Ta

On Tuesday, March 29, 2016, Renato Marroquín Mogrovejo <
renatoj.marroquin@gmail.com> wrote:

> Hey guys,
>
> Sorry for the late reply, I thought that assuming the class of the
> dependency bean should be on the classpath was a fair assumption. So you
> are suggesting something along these lines?
>
> 1. Add another field to the avro schema to point to the dependency class
> location.
> 2. Add the path to the name of the dependency class.
>
> IMHO 2 would be better because avro schemas would still be the same as
> regular avro and not a superset of it. Or what are your thoughts guys?
>
> Best,
>
> Renato M.
> On Mar 24, 2016 10:36 PM, "Lewis John Mcgibbney" <
> lewis.mcgibbney@gmail.com
> <javascript:_e(%7B%7D,'cvml','lewis.mcgibbney@gmail.com');>> wrote:
>
>> Agreed. Will have a crack at a patch for the GoraCompiler and see what is
>> feasible.
>> I believe I also logged a Jora issue or this a while back Henry as this
>> was a blocker for the Gora backend to the OODT file manager component.
>>
>> On Thursday, March 24, 2016, Henry Saputra <henry.saputra@gmail.com
>> <javascript:_e(%7B%7D,'cvml','henry.saputra@gmail.com');>> wrote:
>>
>>> I am not sure how easy it is to add that compiler extension to link to
>>> external Java class from Avro def file.
>>>
>>> If not hard, maybe that is the cleanest way to do it.
>>>
>>> - Henry
>>>
>>> On Wed, Mar 23, 2016 at 10:17 PM, Lewis John Mcgibbney <
>>> lewis.mcgibbney@gmail.com> wrote:
>>>
>>>> My previous discussion on user@avro for reference
>>>> http://www.mail-archive.com/user%40avro.apache.org/msg02488.html
>>>> Would still be interesting to see what the comments are like over here
>>>> if anyone has some,
>>>> Thanks
>>>>
>>>> On Wed, Mar 23, 2016 at 8:45 PM, Lewis John Mcgibbney <
>>>> lewis.mcgibbney@gmail.com> wrote:
>>>>
>>>>> Hi Folks,
>>>>> I'm close to finishing off my long overdue patch for adding Gora
>>>>> support to Log4J 2.X [0] however I have a question which I hope I've
>>>>> explained well enough.
>>>>> In my implementation, the 'object' I am modeling is a Gora instance of
>>>>> the Log4j 2.X NoSqlObject [1].
>>>>> As you can see this object is relatively simple with numerous set(key,
>>>>> value) methods where keys are always strings.
>>>>> The data modeling issue I am having is how to model the complex value
>>>>> if it is, for example, an Object, Object[ ], NoSqlObject or NoSqlObject[ ]?
>>>>> I _think_ I encountered a similar issue a long time aho and turned to
>>>>> Avro IDL to sort the issue out however before I do that I was thinking
>>>>> something along the lines of the following
>>>>>
>>>>> {
>>>>>   "name": "GoraNoSqlObject",
>>>>>   "type": "record",
>>>>>   "namespace": "org.apache.logging.log4j.nosql.appender.gora",
>>>>>   "doc": "GoraNoSqlObject represents the Gora implementation of NoSqlObject.",
>>>>>   "fields": [
>>>>>     {
>>>>>       "name": "keySimpleValue",
>>>>>       "type": {
>>>>>         "type": "map",
>>>>>         "values": ["null","NoSqlObject"]
>>>>>       },
>>>>>       "doc": "Sets the value of a property on this object to a nested complex object.",
>>>>>       "default": {
>>>>>         ...
>>>>>       }
>>>>>     },
>>>>> ...
>>>>>
>>>>>  This however would assume that I could model 'NoSqlObject' elsewhere,
>>>>> and that it had been compiled and available on the GoraCompiler classpath
>>>>> prior to this GoraNoSqlObject class being compiled. To me this just doesn't
>>>>> look right.
>>>>> I just want to get feedback from anyone interested in this to see how
>>>>> they would build the data model.
>>>>> Thanks
>>>>> Lewis
>>>>>
>>>>> [0] https://issues.apache.org/jira/browse/LOG4J2-603
>>>>> [1]
>>>>> https://logging.apache.org/log4j/2.0/log4j-nosql/apidocs/index.html?org/apache/logging/log4j/nosql/appender/NoSqlObject.html
>>>>>
>>>>> --
>>>>> *Lewis*
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> *Lewis*
>>>>
>>>
>>>
>>
>> --
>> *Lewis*
>>
>>

-- 
*Lewis*

Re: Modeling Complex Object Values in Gora

Posted by Renato Marroquín Mogrovejo <re...@gmail.com>.
Hey guys,

Sorry for the late reply, I thought that assuming the class of the
dependency bean should be on the classpath was a fair assumption. So you
are suggesting something along these lines?

1. Add another field to the avro schema to point to the dependency class
location.
2. Add the path to the name of the dependency class.

IMHO 2 would be better because avro schemas would still be the same as
regular avro and not a superset of it. Or what are your thoughts guys?

Best,

Renato M.
On Mar 24, 2016 10:36 PM, "Lewis John Mcgibbney" <le...@gmail.com>
wrote:

> Agreed. Will have a crack at a patch for the GoraCompiler and see what is
> feasible.
> I believe I also logged a Jora issue or this a while back Henry as this
> was a blocker for the Gora backend to the OODT file manager component.
>
> On Thursday, March 24, 2016, Henry Saputra <he...@gmail.com>
> wrote:
>
>> I am not sure how easy it is to add that compiler extension to link to
>> external Java class from Avro def file.
>>
>> If not hard, maybe that is the cleanest way to do it.
>>
>> - Henry
>>
>> On Wed, Mar 23, 2016 at 10:17 PM, Lewis John Mcgibbney <
>> lewis.mcgibbney@gmail.com> wrote:
>>
>>> My previous discussion on user@avro for reference
>>> http://www.mail-archive.com/user%40avro.apache.org/msg02488.html
>>> Would still be interesting to see what the comments are like over here
>>> if anyone has some,
>>> Thanks
>>>
>>> On Wed, Mar 23, 2016 at 8:45 PM, Lewis John Mcgibbney <
>>> lewis.mcgibbney@gmail.com> wrote:
>>>
>>>> Hi Folks,
>>>> I'm close to finishing off my long overdue patch for adding Gora
>>>> support to Log4J 2.X [0] however I have a question which I hope I've
>>>> explained well enough.
>>>> In my implementation, the 'object' I am modeling is a Gora instance of
>>>> the Log4j 2.X NoSqlObject [1].
>>>> As you can see this object is relatively simple with numerous set(key,
>>>> value) methods where keys are always strings.
>>>> The data modeling issue I am having is how to model the complex value
>>>> if it is, for example, an Object, Object[ ], NoSqlObject or NoSqlObject[ ]?
>>>> I _think_ I encountered a similar issue a long time aho and turned to
>>>> Avro IDL to sort the issue out however before I do that I was thinking
>>>> something along the lines of the following
>>>>
>>>> {
>>>>   "name": "GoraNoSqlObject",
>>>>   "type": "record",
>>>>   "namespace": "org.apache.logging.log4j.nosql.appender.gora",
>>>>   "doc": "GoraNoSqlObject represents the Gora implementation of NoSqlObject.",
>>>>   "fields": [
>>>>     {
>>>>       "name": "keySimpleValue",
>>>>       "type": {
>>>>         "type": "map",
>>>>         "values": ["null","NoSqlObject"]
>>>>       },
>>>>       "doc": "Sets the value of a property on this object to a nested complex object.",
>>>>       "default": {
>>>>         ...
>>>>       }
>>>>     },
>>>> ...
>>>>
>>>>  This however would assume that I could model 'NoSqlObject' elsewhere,
>>>> and that it had been compiled and available on the GoraCompiler classpath
>>>> prior to this GoraNoSqlObject class being compiled. To me this just doesn't
>>>> look right.
>>>> I just want to get feedback from anyone interested in this to see how
>>>> they would build the data model.
>>>> Thanks
>>>> Lewis
>>>>
>>>> [0] https://issues.apache.org/jira/browse/LOG4J2-603
>>>> [1]
>>>> https://logging.apache.org/log4j/2.0/log4j-nosql/apidocs/index.html?org/apache/logging/log4j/nosql/appender/NoSqlObject.html
>>>>
>>>> --
>>>> *Lewis*
>>>>
>>>
>>>
>>>
>>> --
>>> *Lewis*
>>>
>>
>>
>
> --
> *Lewis*
>
>

Re: Modeling Complex Object Values in Gora

Posted by Lewis John Mcgibbney <le...@gmail.com>.
Agreed. Will have a crack at a patch for the GoraCompiler and see what is
feasible.
I believe I also logged a Jora issue or this a while back Henry as this was
a blocker for the Gora backend to the OODT file manager component.

On Thursday, March 24, 2016, Henry Saputra <he...@gmail.com> wrote:

> I am not sure how easy it is to add that compiler extension to link to
> external Java class from Avro def file.
>
> If not hard, maybe that is the cleanest way to do it.
>
> - Henry
>
> On Wed, Mar 23, 2016 at 10:17 PM, Lewis John Mcgibbney <
> lewis.mcgibbney@gmail.com
> <javascript:_e(%7B%7D,'cvml','lewis.mcgibbney@gmail.com');>> wrote:
>
>> My previous discussion on user@avro for reference
>> http://www.mail-archive.com/user%40avro.apache.org/msg02488.html
>> Would still be interesting to see what the comments are like over here if
>> anyone has some,
>> Thanks
>>
>> On Wed, Mar 23, 2016 at 8:45 PM, Lewis John Mcgibbney <
>> lewis.mcgibbney@gmail.com
>> <javascript:_e(%7B%7D,'cvml','lewis.mcgibbney@gmail.com');>> wrote:
>>
>>> Hi Folks,
>>> I'm close to finishing off my long overdue patch for adding Gora support
>>> to Log4J 2.X [0] however I have a question which I hope I've explained well
>>> enough.
>>> In my implementation, the 'object' I am modeling is a Gora instance of
>>> the Log4j 2.X NoSqlObject [1].
>>> As you can see this object is relatively simple with numerous set(key,
>>> value) methods where keys are always strings.
>>> The data modeling issue I am having is how to model the complex value if
>>> it is, for example, an Object, Object[ ], NoSqlObject or NoSqlObject[ ]?
>>> I _think_ I encountered a similar issue a long time aho and turned to
>>> Avro IDL to sort the issue out however before I do that I was thinking
>>> something along the lines of the following
>>>
>>> {
>>>   "name": "GoraNoSqlObject",
>>>   "type": "record",
>>>   "namespace": "org.apache.logging.log4j.nosql.appender.gora",
>>>   "doc": "GoraNoSqlObject represents the Gora implementation of NoSqlObject.",
>>>   "fields": [
>>>     {
>>>       "name": "keySimpleValue",
>>>       "type": {
>>>         "type": "map",
>>>         "values": ["null","NoSqlObject"]
>>>       },
>>>       "doc": "Sets the value of a property on this object to a nested complex object.",
>>>       "default": {
>>>         ...
>>>       }
>>>     },
>>> ...
>>>
>>>  This however would assume that I could model 'NoSqlObject' elsewhere,
>>> and that it had been compiled and available on the GoraCompiler classpath
>>> prior to this GoraNoSqlObject class being compiled. To me this just doesn't
>>> look right.
>>> I just want to get feedback from anyone interested in this to see how
>>> they would build the data model.
>>> Thanks
>>> Lewis
>>>
>>> [0] https://issues.apache.org/jira/browse/LOG4J2-603
>>> [1]
>>> https://logging.apache.org/log4j/2.0/log4j-nosql/apidocs/index.html?org/apache/logging/log4j/nosql/appender/NoSqlObject.html
>>>
>>> --
>>> *Lewis*
>>>
>>
>>
>>
>> --
>> *Lewis*
>>
>
>

-- 
*Lewis*

Re: Modeling Complex Object Values in Gora

Posted by Henry Saputra <he...@gmail.com>.
I am not sure how easy it is to add that compiler extension to link to
external Java class from Avro def file.

If not hard, maybe that is the cleanest way to do it.

- Henry

On Wed, Mar 23, 2016 at 10:17 PM, Lewis John Mcgibbney <
lewis.mcgibbney@gmail.com> wrote:

> My previous discussion on user@avro for reference
> http://www.mail-archive.com/user%40avro.apache.org/msg02488.html
> Would still be interesting to see what the comments are like over here if
> anyone has some,
> Thanks
>
> On Wed, Mar 23, 2016 at 8:45 PM, Lewis John Mcgibbney <
> lewis.mcgibbney@gmail.com> wrote:
>
>> Hi Folks,
>> I'm close to finishing off my long overdue patch for adding Gora support
>> to Log4J 2.X [0] however I have a question which I hope I've explained well
>> enough.
>> In my implementation, the 'object' I am modeling is a Gora instance of
>> the Log4j 2.X NoSqlObject [1].
>> As you can see this object is relatively simple with numerous set(key,
>> value) methods where keys are always strings.
>> The data modeling issue I am having is how to model the complex value if
>> it is, for example, an Object, Object[ ], NoSqlObject or NoSqlObject[ ]?
>> I _think_ I encountered a similar issue a long time aho and turned to
>> Avro IDL to sort the issue out however before I do that I was thinking
>> something along the lines of the following
>>
>> {
>>   "name": "GoraNoSqlObject",
>>   "type": "record",
>>   "namespace": "org.apache.logging.log4j.nosql.appender.gora",
>>   "doc": "GoraNoSqlObject represents the Gora implementation of NoSqlObject.",
>>   "fields": [
>>     {
>>       "name": "keySimpleValue",
>>       "type": {
>>         "type": "map",
>>         "values": ["null","NoSqlObject"]
>>       },
>>       "doc": "Sets the value of a property on this object to a nested complex object.",
>>       "default": {
>>         ...
>>       }
>>     },
>> ...
>>
>>  This however would assume that I could model 'NoSqlObject' elsewhere,
>> and that it had been compiled and available on the GoraCompiler classpath
>> prior to this GoraNoSqlObject class being compiled. To me this just doesn't
>> look right.
>> I just want to get feedback from anyone interested in this to see how
>> they would build the data model.
>> Thanks
>> Lewis
>>
>> [0] https://issues.apache.org/jira/browse/LOG4J2-603
>> [1]
>> https://logging.apache.org/log4j/2.0/log4j-nosql/apidocs/index.html?org/apache/logging/log4j/nosql/appender/NoSqlObject.html
>>
>> --
>> *Lewis*
>>
>
>
>
> --
> *Lewis*
>

Re: Modeling Complex Object Values in Gora

Posted by Lewis John Mcgibbney <le...@gmail.com>.
My previous discussion on user@avro for reference
http://www.mail-archive.com/user%40avro.apache.org/msg02488.html
Would still be interesting to see what the comments are like over here if
anyone has some,
Thanks

On Wed, Mar 23, 2016 at 8:45 PM, Lewis John Mcgibbney <
lewis.mcgibbney@gmail.com> wrote:

> Hi Folks,
> I'm close to finishing off my long overdue patch for adding Gora support
> to Log4J 2.X [0] however I have a question which I hope I've explained well
> enough.
> In my implementation, the 'object' I am modeling is a Gora instance of the
> Log4j 2.X NoSqlObject [1].
> As you can see this object is relatively simple with numerous set(key,
> value) methods where keys are always strings.
> The data modeling issue I am having is how to model the complex value if
> it is, for example, an Object, Object[ ], NoSqlObject or NoSqlObject[ ]?
> I _think_ I encountered a similar issue a long time aho and turned to Avro
> IDL to sort the issue out however before I do that I was thinking something
> along the lines of the following
>
> {
>   "name": "GoraNoSqlObject",
>   "type": "record",
>   "namespace": "org.apache.logging.log4j.nosql.appender.gora",
>   "doc": "GoraNoSqlObject represents the Gora implementation of NoSqlObject.",
>   "fields": [
>     {
>       "name": "keySimpleValue",
>       "type": {
>         "type": "map",
>         "values": ["null","NoSqlObject"]
>       },
>       "doc": "Sets the value of a property on this object to a nested complex object.",
>       "default": {
>         ...
>       }
>     },
> ...
>
>  This however would assume that I could model 'NoSqlObject' elsewhere, and
> that it had been compiled and available on the GoraCompiler classpath prior
> to this GoraNoSqlObject class being compiled. To me this just doesn't look
> right.
> I just want to get feedback from anyone interested in this to see how they
> would build the data model.
> Thanks
> Lewis
>
> [0] https://issues.apache.org/jira/browse/LOG4J2-603
> [1]
> https://logging.apache.org/log4j/2.0/log4j-nosql/apidocs/index.html?org/apache/logging/log4j/nosql/appender/NoSqlObject.html
>
> --
> *Lewis*
>



-- 
*Lewis*