You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gora.apache.org by "Julien Nioche (JIRA)" <ji...@apache.org> on 2012/10/26 22:25:11 UTC
[jira] [Created] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Julien Nioche created GORA-174:
----------------------------------
Summary: GORA compiler does not handle ["string", "null"] unions in the AVRO schema
Key: GORA-174
URL: https://issues.apache.org/jira/browse/GORA-174
Project: Apache Gora
Issue Type: Bug
Components: schema
Affects Versions: 0.2.1
Reporter: Julien Nioche
See NUTCH-1477 for description.
We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alfonso Nishikawa reassigned GORA-174:
--------------------------------------
Assignee: Alfonso Nishikawa
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500541#comment-13500541 ]
Alfonso Nishikawa edited comment on GORA-174 at 11/19/12 7:51 PM:
------------------------------------------------------------------
@Henry: yes, that is :)
GORA-174.patch is the fix, while GORA-174-test.patch is the patch for JUnit uploaded by Julien. Now I am trying to understand the JUnit part of the project (takes me some time :P)
was (Author: alfonso.nishikawa):
GORA-174.patch is the fix, while GORA-174-test.patch is the patch for JUnit uploaded by Julien. Now I am trying to understand the JUnit part of the project (takes me some time :P)
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174.patch, GORA-174-test.patch, patch-tests_applied.out.tgz
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500180#comment-13500180 ]
Alfonso Nishikawa commented on GORA-174:
----------------------------------------
Does it work? Doesn't it work? Now still gives the same exception?
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174.patch, GORA-174-test.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alfonso Nishikawa updated GORA-174:
-----------------------------------
Attachment: (was: patch-tests_applied.out.tgz)
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174.patch, GORA-174-test.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alfonso Nishikawa updated GORA-174:
-----------------------------------
Attachment: GORA-174.patch
Patch uploaded. @Julien: try it and tell if it works for you.
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Julien Nioche (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13488566#comment-13488566 ]
Julien Nioche commented on GORA-174:
------------------------------------
anyone left in the project who understands the intricacies of the compiler?
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500635#comment-13500635 ]
Alfonso Nishikawa commented on GORA-174:
----------------------------------------
@Julien: You are right, now seems to (not)work in Eclipse as you say. Thanks.
Actually what your test shows is that a mandatory field fails if you write a null. Otherwise would be optional. At the same time will show that an optional field is optional, but that is not something 100% pedendent on the compiler :P But many many thanks! :) I was getting mad with this tests.
I will finish fixing the tests, since there are other places where CONTENT[i] (=null) is being accessed as an array.
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174.patch, GORA-174-test.patch, patch-tests_applied.out.tgz
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501240#comment-13501240 ]
Alfonso Nishikawa edited comment on GORA-174 at 11/20/12 3:55 PM:
------------------------------------------------------------------
-- deleted
was (Author: alfonso.nishikawa):
Hold a while, I am checking a complex schema. Maybe some almost-unnoticiable bug (writes 'long' where should be 'Long').
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174-test.patch, GORA-174v2.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501240#comment-13501240 ]
Alfonso Nishikawa commented on GORA-174:
----------------------------------------
Hold a while, I am checking a complex schema. Maybe some almost-unnoticiable bug (writes 'long' where should be 'Long').
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174-test.patch, GORA-174v2.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alfonso Nishikawa updated GORA-174:
-----------------------------------
Attachment: GORA-174.patch
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500541#comment-13500541 ]
Alfonso Nishikawa edited comment on GORA-174 at 11/19/12 7:35 PM:
------------------------------------------------------------------
GORA-174.patch is the fix, while GORA-174-test.patch is the patch for JUnit uploaded by Julien. Now I am trying to understand the JUnit part of the project (takes me some time :P)
was (Author: alfonso.nishikawa):
GORA-174.patch is the fix, while GORA-174-test.patch is the patch for JUnit uploaded by Julien. Now I am trying to understand the JUnit part of the proyect (takes me some time :P)
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174.patch, GORA-174-test.patch, patch-tests_applied.out.tgz
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503410#comment-13503410 ]
Alfonso Nishikawa commented on GORA-174:
----------------------------------------
Patch is for gora 0.2.1 . Will adapt it to /trunk and upload again.
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174-test.patch, GORA-174v2.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alfonso Nishikawa updated GORA-174:
-----------------------------------
Attachment: failed_tests_after_v3.tar.gz
Uploaded failed tests after applying GORA-174v3.patch
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: failed_tests_after_v3.tar.gz, GORA-174-test.patch, GORA-174v2.patch, GORA-174v3.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alfonso Nishikawa updated GORA-174:
-----------------------------------
Attachment: GORA-174v3.patch
========= patch released =============
GORA-174v3.patch
Released for /trunk although not much differences.
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174-test.patch, GORA-174v2.patch, GORA-174v3.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500338#comment-13500338 ]
Alfonso Nishikawa commented on GORA-174:
----------------------------------------
I applied your patch and passed tests of JUnit in Eclipse (org.apache.gora.avro.store.TestDataFileAvroStore) :P
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174.patch, GORA-174-test.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alfonso Nishikawa updated GORA-174:
-----------------------------------
Attachment: (was: GORA-174.patch)
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Lewis John McGibbney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509916#comment-13509916 ]
Lewis John McGibbney commented on GORA-174:
-------------------------------------------
I can confirm that after applying the patch to gora-core then using this dependency with Nutch 2.x, the Exception is identical as the one documented in NUTCH-1477. When I compile the webpage avsc's here [0], the webpage class is generated but with no accessors for the fields in with the union case we are concerned with. I am also learning with the GoraCompiler (and Avro stuff) so I do not have a definitive solution to hand just now. I think the serializing and deserializing problems which seem to be a result of introducing union support should be addressed as we encounter them.
[0] http://svn.apache.org/repos/asf/nutch/branches/2.x/src/gora/
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: failed_tests_after_v3.tar.gz, GORA-174-test.patch, GORA-174v2.patch, GORA-174v3.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alfonso Nishikawa updated GORA-174:
-----------------------------------
Attachment: GORA-174.patch
Generate getters/setters for union types.
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Fix For: 0.3
>
> Attachments: GORA-174.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500358#comment-13500358 ]
Alfonso Nishikawa edited comment on GORA-174 at 11/19/12 4:49 PM:
------------------------------------------------------------------
Oh, oh, sorry :P applied both patches :P
I'm doing it wrong. Will try it again. Maybe I generated the classes by command. I am with other things too and don't remember :'(
was (Author: alfonso.nishikawa):
Oh, oh, sorry :P applied both patches :P
I'm doing it wrong. Will try it again.
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174.patch, GORA-174-test.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500098#comment-13500098 ]
Alfonso Nishikawa commented on GORA-174:
----------------------------------------
Hi, Julien. Hold a second. Found some minor bug that stops compiling.
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alfonso Nishikawa updated GORA-174:
-----------------------------------
Attachment: patch-tests_applied.out.tgz
Output of mvn clean test uploaded (patch-tests_applied). Only applied "GORA-174-test.patch".
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174.patch, GORA-174-test.patch, patch-tests_applied.out.tgz
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13499913#comment-13499913 ]
Alfonso Nishikawa edited comment on GORA-174 at 11/18/12 9:38 PM:
------------------------------------------------------------------
Hi, Julien.
Try the patch and tell me if it solves your problem (I tested, but who knows...).
Sadly, you will get the exception in GORA-183. Will try to fix it soon.
was (Author: alfonso.nishikawa):
Hi, Julien.
Try the patch and tell me if it solves your problem (I tested, but who knows...).
When using optional fields (those unions with "null") you will get the exception in GORA-183 if you leave a field=null. Will try to fix it soon.
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Fix For: 0.3
>
> Attachments: GORA-174.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500541#comment-13500541 ]
Alfonso Nishikawa commented on GORA-174:
----------------------------------------
GORA-174.patch is the fix, while GORA-174-test.patch is the patch for JUnit uploaded by Julien. Now I am trying to understand the JUnit part of the proyect (takes me some time :P)
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174.patch, GORA-174-test.patch, patch-tests_applied.out.tgz
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13499913#comment-13499913 ]
Alfonso Nishikawa commented on GORA-174:
----------------------------------------
Hi, Julien.
Try the patch and tell me if it solves your problem (I tested, but who knows...).
You will find when using optional fields (those unions with "null") that you will get the exception in GORA-183 if you leave a field=null. Will try to fix it soon.
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Fix For: 0.3
>
> Attachments: GORA-174.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500358#comment-13500358 ]
Alfonso Nishikawa commented on GORA-174:
----------------------------------------
Oh, oh, sorry :P applied both patches :P
I'm doing it wrong. Will try it again.
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174.patch, GORA-174-test.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alfonso Nishikawa updated GORA-174:
-----------------------------------
Comment: was deleted
(was: Thanks :) hope it works!)
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174-test.patch, GORA-174v2.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500181#comment-13500181 ]
Alfonso Nishikawa commented on GORA-174:
----------------------------------------
Ah, ok, thanks for the information :) DataFileAvroStore, didn't notice.
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174.patch, GORA-174-test.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Julien Nioche (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500329#comment-13500329 ]
Julien Nioche commented on GORA-174:
------------------------------------
Do you mean that it does not fail when calling 'mvn clean test'? If so did you regenerate the java code from the modified AVRO schema manually?
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174.patch, GORA-174-test.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Lewis John McGibbney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501229#comment-13501229 ]
Lewis John McGibbney commented on GORA-174:
-------------------------------------------
Hi, I'll try the new patch both with gora trunk and with the Nutch 2.x InjectorJob.
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174-test.patch, GORA-174v2.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alfonso Nishikawa updated GORA-174:
-----------------------------------
Comment: was deleted
(was: Generate getters/setters for union types.)
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Fix For: 0.3
>
> Attachments: GORA-174.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Julien Nioche (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500595#comment-13500595 ]
Julien Nioche commented on GORA-174:
------------------------------------
I just took a fresh copy from the GIT repo, applied the patch GORA-174-test.patch, cd gora-core and called 'mvn clean test' and I getting
Failed tests: testQuery(org.apache.gora.avro.store.TestAvroStore): expected:<0> but was:<10>
testQueryBinaryEncoder(org.apache.gora.avro.store.TestAvroStore): expected:<0> but was:<10>
testQuery(org.apache.gora.avro.store.TestDataFileAvroStore): expected:<0> but was:<10>
testQueryBinaryEncoder(org.apache.gora.avro.store.TestDataFileAvroStore): expected:<0> but was:<10>
Tests in error:
testCountQuery(org.apache.gora.avro.mapreduce.TestDataFileAvroStoreMapReduce)
Running the tests without applying the patch GORA-174-test.patch works fine.
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174.patch, GORA-174-test.patch, patch-tests_applied.out.tgz
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alfonso Nishikawa updated GORA-174:
-----------------------------------
Comment: was deleted
(was: Output of mvn clean test uploaded (patch-tests_applied). Only applied "GORA-174-test.patch".)
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174.patch, GORA-174-test.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500575#comment-13500575 ]
Alfonso Nishikawa commented on GORA-174:
----------------------------------------
@Julien: after analyzing your test classes patch, I think they work as expected: before your changes, the class WebPage exists, and method WebPage#setContent() too. After your patch, without regenerating the class WebPage, the method WebPage#setContent() too, so WebPage#setContent(null) will not fail. DataFileAvroStore success writing a null. No problem.
To make it fail must: modify webpage.json, regenerate WebPage, and will not compile because the method WebPage#setContent() will not exist.
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174.patch, GORA-174-test.patch, patch-tests_applied.out.tgz
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509872#comment-13509872 ]
Alfonso Nishikawa commented on GORA-174:
----------------------------------------
Hi, Lewis.
Thank you for trying it. Does it give the same exception as in NUTCH-1477?
Adding union support in the compiler seems to affect serializing and deserializing too (and others - added failed tests in attachments).
As you told in other thread, this seems to become a pain. I subscribe ;)
I am working on this issue and will be working all this week :) Suggestions much accepted and looking forward them.
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: failed_tests_after_v3.tar.gz, GORA-174-test.patch, GORA-174v2.patch, GORA-174v3.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13499913#comment-13499913 ]
Alfonso Nishikawa edited comment on GORA-174 at 11/18/12 11:41 PM:
-------------------------------------------------------------------
Hi, Julien.
Try the patch and tell me if it solves your problem (I tested, but who knows...).
Sadly, you will get the exception in GORA-183. Will try to fix it soon.
--edited: GORA-183 affects HBase backend. What are you using?
was (Author: alfonso.nishikawa):
Hi, Julien.
Try the patch and tell me if it solves your problem (I tested, but who knows...).
Sadly, you will get the exception in GORA-183. Will try to fix it soon.
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Fix For: 0.3
>
> Attachments: GORA-174.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501250#comment-13501250 ]
Alfonso Nishikawa commented on GORA-174:
----------------------------------------
Seems all right :) Thanks!! Hope it works with Nutch.
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174-test.patch, GORA-174v2.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501239#comment-13501239 ]
Alfonso Nishikawa commented on GORA-174:
----------------------------------------
Thanks :) hope it works!
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174-test.patch, GORA-174v2.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Henry Saputra (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500531#comment-13500531 ]
Henry Saputra commented on GORA-174:
------------------------------------
So GORA-174-test.patch file is the right patch to try?
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174.patch, GORA-174-test.patch, patch-tests_applied.out.tgz
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Julien Nioche (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500179#comment-13500179 ]
Julien Nioche commented on GORA-174:
------------------------------------
DataFileAvroStore - see description of this issue and of NUTCH-1477
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174.patch, GORA-174-test.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Julien Nioche (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Julien Nioche updated GORA-174:
-------------------------------
Attachment: GORA-174-test.patch
Will test your patch with the Nutch schema. In the meantime here is a patch for the tests that currently crashes when we add a null value to the schema. For some reason the java code is not generated prior to running the tests. Hopefully this test should pass after fixing this issue.
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174.patch, GORA-174-test.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alfonso Nishikawa updated GORA-174:
-----------------------------------
Attachment: (was: GORA-174.patch)
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Lewis John McGibbney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lewis John McGibbney updated GORA-174:
--------------------------------------
Fix Version/s: 0.3
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Fix For: 0.3
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500172#comment-13500172 ]
Alfonso Nishikawa commented on GORA-174:
----------------------------------------
Remember that in HBase will not work (issue GORA-183). What backend are you using?
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174.patch, GORA-174-test.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Henry Saputra (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13508948#comment-13508948 ]
Henry Saputra commented on GORA-174:
------------------------------------
+1 for closing GORA-183 as dup of GORA-174
Looks like the fix for GORA-174 should be able to resolve the issue. Alfonso, if thats correct please close GORA-183 as dup
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: failed_tests_after_v3.tar.gz, GORA-174-test.patch, GORA-174v2.patch, GORA-174v3.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Lewis John McGibbney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509812#comment-13509812 ]
Lewis John McGibbney commented on GORA-174:
-------------------------------------------
Hi Alfonso, at the beginning of the weekend I spent time working between the patched GoraCompiler and Nutch 2.x. AS Julien suggested in his initial problem description "however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling", this still seems to be the case even when using the GORA-174v3.patch so I am not sure that the patch is properly fixing this issue.
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: failed_tests_after_v3.tar.gz, GORA-174-test.patch, GORA-174v2.patch, GORA-174v3.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alfonso Nishikawa updated GORA-174:
-----------------------------------
Attachment: (was: GORA-174.patch)
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13508864#comment-13508864 ]
Alfonso Nishikawa commented on GORA-174:
----------------------------------------
Checked all tests. "mvn test" shows all passed, but checking gora-xxx/target/surefire-reports/ realized there where the following failed tests:
* Gora-core: org.apache.gora.mapreduce.TestPersistentSerialization - testSerdeWebPage. GORA-188
* Gora-accumulo: org.apache.gora.accumulo.store.AccumuloStoreTest - testUpdate. java.lang.IllegalArgumentException: Unknown type UNION and testing classes related with "null" values.
* Gora-cassanda: NPE related with "null" values.
* Gora-hbase: [java.lang.IllegalArgumentException: Unknown type UNION, NPE, ... shown in GORA-183
Maybe we should close GORA-183 and solve it here.
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: failed_tests_after_v3.tar.gz, GORA-174-test.patch, GORA-174v2.patch, GORA-174v3.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Julien Nioche (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500361#comment-13500361 ]
Julien Nioche commented on GORA-174:
------------------------------------
No probs. Even if you apply both patches the mvn tests will still fail - hence my comments about the code generation above.
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174.patch, GORA-174-test.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alfonso Nishikawa updated GORA-174:
-----------------------------------
Attachment: GORA-174.patch
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Julien Nioche (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500350#comment-13500350 ]
Julien Nioche commented on GORA-174:
------------------------------------
I don't know why it does not fail with Eclipse but the testing should be done with 'mvn clean test' and in this particular case we DO want the tests to fail as a proof that we need your patch and that it works.
Would be great to clarify what generates the Java code from the AVRO schemas in gora-core/src/examples/avro and when. Thoughts anyone?
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174.patch, GORA-174-test.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13499913#comment-13499913 ]
Alfonso Nishikawa edited comment on GORA-174 at 11/18/12 9:35 PM:
------------------------------------------------------------------
Hi, Julien.
Try the patch and tell me if it solves your problem (I tested, but who knows...).
When using optional fields (those unions with "null") you will get the exception in GORA-183 if you leave a field=null. Will try to fix it soon.
was (Author: alfonso.nishikawa):
Hi, Julien.
Try the patch and tell me if it solves your problem (I tested, but who knows...).
You will find when using optional fields (those unions with "null") that you will get the exception in GORA-183 if you leave a field=null. Will try to fix it soon.
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Fix For: 0.3
>
> Attachments: GORA-174.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Alfonso Nishikawa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500278#comment-13500278 ]
Alfonso Nishikawa commented on GORA-174:
----------------------------------------
Thanks for the tests patch.
In my source copy (downloaded .tgz), this line of the patch:
{code}
@@ -116,9 +116,11 @@ public class WebPageDataCreator {
{code}
should be
{code}
@@ -115,9 +115,11 @@ public class WebPageDataCreator {
{code}
but patched successfuly with "patch -p0 -l -i GORA-174-test.patch".
Seems to work for me.
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174.patch, GORA-174-test.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-174) GORA compiler does not handle
["string", "null"] unions in the AVRO schema
Posted by "Julien Nioche (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500183#comment-13500183 ]
Julien Nioche commented on GORA-174:
------------------------------------
bq. Will test your patch with the Nutch schema
no comment means that I have not had the time to test it
you can use the patch I submitted earlier to reproduce the issue yourself within GORA independently of Nutch
> GORA compiler does not handle ["string", "null"] unions in the AVRO schema
> --------------------------------------------------------------------------
>
> Key: GORA-174
> URL: https://issues.apache.org/jira/browse/GORA-174
> Project: Apache Gora
> Issue Type: Bug
> Components: schema
> Affects Versions: 0.2.1
> Reporter: Julien Nioche
> Assignee: Alfonso Nishikawa
> Fix For: 0.3
>
> Attachments: GORA-174.patch, GORA-174-test.patch
>
>
> See NUTCH-1477 for description.
> We are getting NPE when using the DataFileAvroStore, in order to avoid that I modified the schema to allow for null values on some fields e.g. {"name": "baseUrl", "type": ["string", "null"] }
> however when generating the code for the schema the accessors are not generated by GORA which prevents Nutch from compiling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira