You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Eelco Hillenius (JIRA)" <ji...@apache.org> on 2009/08/20 09:09:14 UTC

[jira] Created: (AVRO-104) Reflective API can't handle nulls

Reflective API can't handle nulls
---------------------------------

                 Key: AVRO-104
                 URL: https://issues.apache.org/jira/browse/AVRO-104
             Project: Avro
          Issue Type: Bug
          Components: java
    Affects Versions: 1.0.0, 1.0.1
            Reporter: Eelco Hillenius
             Fix For: 1.0.1
         Attachments: AVRO-104.patch

It is currently not possible to have fields with null values when using the reflective API.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (AVRO-104) Reflective API can't handle nulls

Posted by "Eelco Hillenius (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/AVRO-104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747013#action_12747013 ] 

Eelco Hillenius commented on AVRO-104:
--------------------------------------

I'd be happy taking a stab at that. However, it seems that there is currently a deeper problem... even when creating fields as unions of their type and nulls, it doesn't work. It seems their values are always written out as nulls. As you can see when you run the unit test with the last patch.

> Reflective API can't handle nulls
> ---------------------------------
>
>                 Key: AVRO-104
>                 URL: https://issues.apache.org/jira/browse/AVRO-104
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.0.0, 1.0.1
>            Reporter: Eelco Hillenius
>             Fix For: 1.0.1
>
>         Attachments: AVRO-104.patch, AVRO-104.patch, AVRO-104.patch
>
>
> It is currently not possible to have fields with null values when using the reflective API.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (AVRO-104) Reflective API can't handle nulls

Posted by "Eelco Hillenius (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/AVRO-104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eelco Hillenius updated AVRO-104:
---------------------------------

    Attachment: AVRO-104.patch

Uploaded patch that gets rid of the static methods of ReflectData in favor of using - now overridable - instances of ReflectData.

For convenience (and because I would expect more people to want to allow nulls when writing records through introspection, I'm exposing the null allowing specialization of ReflectData like this:

ReflectData reflectData = ReflectData.newNullAllowingInstance();

But people can provide any custom implementation.


> Reflective API can't handle nulls
> ---------------------------------
>
>                 Key: AVRO-104
>                 URL: https://issues.apache.org/jira/browse/AVRO-104
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.0.0, 1.0.1
>            Reporter: Eelco Hillenius
>             Fix For: 1.0.1
>
>         Attachments: AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch
>
>
> It is currently not possible to have fields with null values when using the reflective API.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (AVRO-104) Reflective API can't handle nulls

Posted by "Eelco Hillenius (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/AVRO-104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eelco Hillenius updated AVRO-104:
---------------------------------

    Attachment: AVRO-104.patch

Added a test that uses ReflectDatumWriter and found a bug in the process. Had to make schema cache non static. Clients can always cache their own ReflectData instance.

> Reflective API can't handle nulls
> ---------------------------------
>
>                 Key: AVRO-104
>                 URL: https://issues.apache.org/jira/browse/AVRO-104
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.0.0, 1.0.1
>            Reporter: Eelco Hillenius
>             Fix For: 1.0.1
>
>         Attachments: AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch
>
>
> It is currently not possible to have fields with null values when using the reflective API.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (AVRO-104) Reflective API can't handle nulls

Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/AVRO-104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747006#action_12747006 ] 

Doug Cutting commented on AVRO-104:
-----------------------------------

> Otherwise, I think it would be nice to implement reflection through a separate abstraction where you allow clients to plug in their own implementation in Avro.

Sure, one might make ReflectDatumReader and ReflectDatumWriter each contain a ReflectData instance, and add protected methods that create the schemas for each Java type.

> Reflective API can't handle nulls
> ---------------------------------
>
>                 Key: AVRO-104
>                 URL: https://issues.apache.org/jira/browse/AVRO-104
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.0.0, 1.0.1
>            Reporter: Eelco Hillenius
>             Fix For: 1.0.1
>
>         Attachments: AVRO-104.patch, AVRO-104.patch, AVRO-104.patch
>
>
> It is currently not possible to have fields with null values when using the reflective API.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (AVRO-104) Reflective API can't handle nulls

Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/AVRO-104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747669#action_12747669 ] 

Doug Cutting commented on AVRO-104:
-----------------------------------

The command 'ant clean test' fails on trunk with this patch applied:

[checkstyle] /home/cutting/src/avro/work/src/java/org/apache/avro/reflect/ReflectData.java:260:27: Name 'PARANAMER' must match pattern '^[a-z][a-zA-Z0-9]*$'.

Non-constants should not be all-caps.


> Reflective API can't handle nulls
> ---------------------------------
>
>                 Key: AVRO-104
>                 URL: https://issues.apache.org/jira/browse/AVRO-104
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.0.0, 1.0.1
>            Reporter: Eelco Hillenius
>             Fix For: 1.0.1
>
>         Attachments: AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch
>
>
> It is currently not possible to have fields with null values when using the reflective API.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (AVRO-104) Reflective API can't handle nulls

Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/AVRO-104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747536#action_12747536 ] 

Doug Cutting commented on AVRO-104:
-----------------------------------

> The problem is with the static getClient methods of ReflectRequestor, ReflectResponder and SpecificRequestor, which do a static call to ReflectData themselves.

Folks can easily avoid those methods by calling Proxy.newProxyInstance themselves, or we could add a new version of getClient() that accepts a ReflectData implementation.

> Btw, ReflectData#validate is nowhere used nor tested in the Avro code base. Is this a forgotten method (code rot), or does it just need a test case? 

Somewhere between.  We should probably add a test.  Can you think of cases where it'd be handy to have?

> Reflective API can't handle nulls
> ---------------------------------
>
>                 Key: AVRO-104
>                 URL: https://issues.apache.org/jira/browse/AVRO-104
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.0.0, 1.0.1
>            Reporter: Eelco Hillenius
>             Fix For: 1.0.1
>
>         Attachments: AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch
>
>
> It is currently not possible to have fields with null values when using the reflective API.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (AVRO-104) Reflective API can't handle nulls

Posted by "Eelco Hillenius (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/AVRO-104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eelco Hillenius updated AVRO-104:
---------------------------------

    Attachment: AVRO-104.patch

Ugh, that was sloppy! Uploaded a new patch that shows that member vars that are null indeed can be supported if they are defined as a union of their type and Avro's NULL type.

> Reflective API can't handle nulls
> ---------------------------------
>
>                 Key: AVRO-104
>                 URL: https://issues.apache.org/jira/browse/AVRO-104
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.0.0, 1.0.1
>            Reporter: Eelco Hillenius
>             Fix For: 1.0.1
>
>         Attachments: AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch
>
>
> It is currently not possible to have fields with null values when using the reflective API.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (AVRO-104) Reflective API can't handle nulls

Posted by "Eelco Hillenius (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/AVRO-104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747609#action_12747609 ] 

Eelco Hillenius commented on AVRO-104:
--------------------------------------

Patch is in, so we have two alternatives now. The latter, while causing a slight constructor explosion, doesn't turn out to be as bad as I thought.

Is this patch acceptable?

> Reflective API can't handle nulls
> ---------------------------------
>
>                 Key: AVRO-104
>                 URL: https://issues.apache.org/jira/browse/AVRO-104
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.0.0, 1.0.1
>            Reporter: Eelco Hillenius
>             Fix For: 1.0.1
>
>         Attachments: AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch
>
>
> It is currently not possible to have fields with null values when using the reflective API.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (AVRO-104) Reflective API can't handle nulls

Posted by "Eelco Hillenius (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/AVRO-104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eelco Hillenius updated AVRO-104:
---------------------------------

    Attachment: AVRO-104.patch

Uploaded patch with alternative solution. Works through a delegate 'ReflectSchemaDelegate' you can set on ReflectData staticaly.

Imho, It would be more elegant to get rid of a lot of the use of static throughout the AVRO API, but that would mean quite a few breaks, and would make using AVRO also more verbose. This patch imho is a reasonable hack; not as ugly as setting schemas for fields directly, and more flexible.

People like me who want to allow null values of fields in objects they let AVRO write out, can do this:

      ReflectData.setReflectSchemaDelegate(
          ReflectSchemaDelegate.newNullAllowingInstance());

and AVRO will create a union for every field it comes across with introspection. Clients can also provide a custom strategy if they wish to do so.

> Reflective API can't handle nulls
> ---------------------------------
>
>                 Key: AVRO-104
>                 URL: https://issues.apache.org/jira/browse/AVRO-104
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.0.0, 1.0.1
>            Reporter: Eelco Hillenius
>             Fix For: 1.0.1
>
>         Attachments: AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch
>
>
> It is currently not possible to have fields with null values when using the reflective API.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (AVRO-104) Reflective API can't handle nulls

Posted by "Eelco Hillenius (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/AVRO-104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747513#action_12747513 ] 

Eelco Hillenius commented on AVRO-104:
--------------------------------------

Btw, ReflectData#validate is nowhere used nor tested in the Avro code base. Is this a forgotten method (code rot), or does it just need a test case?

> Reflective API can't handle nulls
> ---------------------------------
>
>                 Key: AVRO-104
>                 URL: https://issues.apache.org/jira/browse/AVRO-104
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.0.0, 1.0.1
>            Reporter: Eelco Hillenius
>             Fix For: 1.0.1
>
>         Attachments: AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch
>
>
> It is currently not possible to have fields with null values when using the reflective API.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (AVRO-104) Reflective API can't handle nulls

Posted by "Eelco Hillenius (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/AVRO-104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eelco Hillenius updated AVRO-104:
---------------------------------

    Attachment: AVRO-104.patch

Attached patch for TestReflect to demonstrate the issue

> Reflective API can't handle nulls
> ---------------------------------
>
>                 Key: AVRO-104
>                 URL: https://issues.apache.org/jira/browse/AVRO-104
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.0.0, 1.0.1
>            Reporter: Eelco Hillenius
>             Fix For: 1.0.1
>
>         Attachments: AVRO-104.patch
>
>
> It is currently not possible to have fields with null values when using the reflective API.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (AVRO-104) Reflective API can't handle nulls

Posted by "Eelco Hillenius (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/AVRO-104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eelco Hillenius updated AVRO-104:
---------------------------------

    Attachment: AVRO-104.patch

Uploaded patch that removes the null checks and makes PARANAMER non-static.

> Reflective API can't handle nulls
> ---------------------------------
>
>                 Key: AVRO-104
>                 URL: https://issues.apache.org/jira/browse/AVRO-104
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.0.0, 1.0.1
>            Reporter: Eelco Hillenius
>             Fix For: 1.0.1
>
>         Attachments: AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch
>
>
> It is currently not possible to have fields with null values when using the reflective API.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (AVRO-104) Reflective API can't handle nulls

Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/AVRO-104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Doug Cutting updated AVRO-104:
------------------------------

    Resolution: Fixed
      Assignee: Eelco Hillenius
        Status: Resolved  (was: Patch Available)

I just committed this.  Thanks, Eelco!

> Reflective API can't handle nulls
> ---------------------------------
>
>                 Key: AVRO-104
>                 URL: https://issues.apache.org/jira/browse/AVRO-104
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.0.0, 1.0.1
>            Reporter: Eelco Hillenius
>            Assignee: Eelco Hillenius
>             Fix For: 1.0.1
>
>         Attachments: AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch
>
>
> It is currently not possible to have fields with null values when using the reflective API.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (AVRO-104) Reflective API can't handle nulls

Posted by "Eelco Hillenius (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/AVRO-104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747213#action_12747213 ] 

Eelco Hillenius commented on AVRO-104:
--------------------------------------

> Sure, one might make ReflectDatumReader and ReflectDatumWriter each contain a ReflectData instance, and add protected methods that create the schemas for each Java type.

I've looked at supplying a patch that does this, but thanks to the contagious nature of static calls, it would require quite a bit of refactoring. For instance, ReflectRequestor, ReflectResponder and SpecificRequestor getClient methods would need to be made non-static for this to work, and those classes plus ReflectDatumReader/ Writer would need to have the ReflectData instance passed in, since it would be used right away in the constructor, which makes implementing it through a protected (overridable) method (like newReflectData()) dangerous as the object is still being constructed. So it seems that the patch I supply here is the least intrusive, and though not particularly elegant, probably better than the alternative, unless you don't mind breaking a bunch of clients.

> Reflective API can't handle nulls
> ---------------------------------
>
>                 Key: AVRO-104
>                 URL: https://issues.apache.org/jira/browse/AVRO-104
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.0.0, 1.0.1
>            Reporter: Eelco Hillenius
>             Fix For: 1.0.1
>
>         Attachments: AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch
>
>
> It is currently not possible to have fields with null values when using the reflective API.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (AVRO-104) Reflective API can't handle nulls

Posted by "Eelco Hillenius (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/AVRO-104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747545#action_12747545 ] 

Eelco Hillenius commented on AVRO-104:
--------------------------------------

>  or we could add a new version of getClient() that accepts a ReflectData implementation.

Ok. I'll give that a try when I have the chance.

> Somewhere between. We should probably add a test. Can you think of cases where it'd be handy to have?

Not sure. I would think I'd find out quick enough that there are errors when trying to write datum, so I don't have the need for that method personally. Doesn't seem to be a bad thing to have in the API, but then imho it needs a test so that we know that at least it works.

> Reflective API can't handle nulls
> ---------------------------------
>
>                 Key: AVRO-104
>                 URL: https://issues.apache.org/jira/browse/AVRO-104
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.0.0, 1.0.1
>            Reporter: Eelco Hillenius
>             Fix For: 1.0.1
>
>         Attachments: AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch
>
>
> It is currently not possible to have fields with null values when using the reflective API.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (AVRO-104) Reflective API can't handle nulls

Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/AVRO-104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747501#action_12747501 ] 

Doug Cutting commented on AVRO-104:
-----------------------------------

I'd vote for doing the refactoring to make ReflectData non-static.  We can probably continue to support the existing ReflectDatumReader/Writer constructors back-compatibly, no?  If not, we'd put this in 1.1 rather than 1.0.1.


> Reflective API can't handle nulls
> ---------------------------------
>
>                 Key: AVRO-104
>                 URL: https://issues.apache.org/jira/browse/AVRO-104
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.0.0, 1.0.1
>            Reporter: Eelco Hillenius
>             Fix For: 1.0.1
>
>         Attachments: AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch
>
>
> It is currently not possible to have fields with null values when using the reflective API.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (AVRO-104) Reflective API can't handle nulls

Posted by "Eelco Hillenius (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/AVRO-104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eelco Hillenius updated AVRO-104:
---------------------------------

    Attachment: AVRO-104.patch

Final patch I hope: include minor check style fixes

> Reflective API can't handle nulls
> ---------------------------------
>
>                 Key: AVRO-104
>                 URL: https://issues.apache.org/jira/browse/AVRO-104
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.0.0, 1.0.1
>            Reporter: Eelco Hillenius
>             Fix For: 1.0.1
>
>         Attachments: AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch
>
>
> It is currently not possible to have fields with null values when using the reflective API.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (AVRO-104) Reflective API can't handle nulls

Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/AVRO-104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747025#action_12747025 ] 

Doug Cutting commented on AVRO-104:
-----------------------------------

> I think that is a bug, but maybe I'm overlooking something?

I think the bug is in your test case.  You read the same object twice.

> Reflective API can't handle nulls
> ---------------------------------
>
>                 Key: AVRO-104
>                 URL: https://issues.apache.org/jira/browse/AVRO-104
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.0.0, 1.0.1
>            Reporter: Eelco Hillenius
>             Fix For: 1.0.1
>
>         Attachments: AVRO-104.patch, AVRO-104.patch, AVRO-104.patch
>
>
> It is currently not possible to have fields with null values when using the reflective API.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (AVRO-104) Reflective API can't handle nulls

Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/AVRO-104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747620#action_12747620 ] 

Doug Cutting commented on AVRO-104:
-----------------------------------

A few minor nits:
 - Paranamer might not be thread safe, so it should probably not be static.
 - The null checks seem excessive and not that informative.  Folks will fail soon enough if they pass a null here and this will be caught in debugging, long before production.  None of the other parameters are null-checked.

Other than that, this looks good to me.


> Reflective API can't handle nulls
> ---------------------------------
>
>                 Key: AVRO-104
>                 URL: https://issues.apache.org/jira/browse/AVRO-104
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.0.0, 1.0.1
>            Reporter: Eelco Hillenius
>             Fix For: 1.0.1
>
>         Attachments: AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch
>
>
> It is currently not possible to have fields with null values when using the reflective API.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


contributing

Posted by Doug Cutting <cu...@apache.org>.
dane van dyck wrote:
> Sorry to bother you all, but I'm interested in contributing to Avro, and 
> am seeking a little more information about the state of the project. Are 
> any of you interested in advising me on areas which might be accessible 
> to a newcomer? If not, please redirect me towards another resource.

The best way to get involved is to try to use it, see what you find 
lacking, and then try to help fix it.

The following Wiki page has some hints about the mechanics of getting 
started:

http://wiki.apache.org/hadoop/Avro/HowToContribute

If you have more questions, please ask!

Doug

Re: [jira] Commented: (AVRO-104) Reflective API can't handle nulls

Posted by dane van dyck <dn...@gmail.com>.
Eelco Hillenius (JIRA) wrote:
>     [ https://issues.apache.org/jira/browse/AVRO-104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747512#action_12747512 ] 
>
> Eelco Hillenius commented on AVRO-104:
> --------------------------------------
>
> ReflectDatumReader/Writer could be done by adding more constructors and setting ReflectData as the default. The problem is with the static getClient methods of ReflectRequestor, ReflectResponder and SpecificRequestor, which do a static call to ReflectData themselves.
>
> We might be possible is to keep ReflectData#getProtocol static, and getSchema or createSchema overridable as non-static methods. Would that be acceptable?
>
>   
>> Reflective API can't handle nulls
>> ---------------------------------
>>
>>                 Key: AVRO-104
>>                 URL: https://issues.apache.org/jira/browse/AVRO-104
>>             Project: Avro
>>          Issue Type: Bug
>>          Components: java
>>    Affects Versions: 1.0.0, 1.0.1
>>            Reporter: Eelco Hillenius
>>             Fix For: 1.0.1
>>
>>         Attachments: AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch
>>
>>
>> It is currently not possible to have fields with null values when using the reflective API.
>>     
>
>   
Sorry to bother you all, but I'm interested in contributing to Avro, and 
am seeking a little more information about the state of the project. Are 
any of you interested in advising me on areas which might be accessible 
to a newcomer? If not, please redirect me towards another resource.

Thanks,
Dane VanDyck

[jira] Commented: (AVRO-104) Reflective API can't handle nulls

Posted by "Eelco Hillenius (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/AVRO-104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747512#action_12747512 ] 

Eelco Hillenius commented on AVRO-104:
--------------------------------------

ReflectDatumReader/Writer could be done by adding more constructors and setting ReflectData as the default. The problem is with the static getClient methods of ReflectRequestor, ReflectResponder and SpecificRequestor, which do a static call to ReflectData themselves.

We might be possible is to keep ReflectData#getProtocol static, and getSchema or createSchema overridable as non-static methods. Would that be acceptable?

> Reflective API can't handle nulls
> ---------------------------------
>
>                 Key: AVRO-104
>                 URL: https://issues.apache.org/jira/browse/AVRO-104
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.0.0, 1.0.1
>            Reporter: Eelco Hillenius
>             Fix For: 1.0.1
>
>         Attachments: AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch
>
>
> It is currently not possible to have fields with null values when using the reflective API.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (AVRO-104) Reflective API can't handle nulls

Posted by "Eelco Hillenius (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/AVRO-104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eelco Hillenius updated AVRO-104:
---------------------------------

    Status: Patch Available  (was: Open)

> Reflective API can't handle nulls
> ---------------------------------
>
>                 Key: AVRO-104
>                 URL: https://issues.apache.org/jira/browse/AVRO-104
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.0.0, 1.0.1
>            Reporter: Eelco Hillenius
>             Fix For: 1.0.1
>
>         Attachments: AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch, AVRO-104.patch
>
>
> It is currently not possible to have fields with null values when using the reflective API.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.