You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Ron Bodkin (JIRA)" <ji...@apache.org> on 2010/09/19 02:12:33 UTC
[jira] Created: (AVRO-671) SpecificCompiler generates invalid Java
code for invalid identifiers
SpecificCompiler generates invalid Java code for invalid identifiers
--------------------------------------------------------------------
Key: AVRO-671
URL: https://issues.apache.org/jira/browse/AVRO-671
Project: Avro
Issue Type: Bug
Reporter: Ron Bodkin
e.g., if you have a field named "a.b" it will attempt to generate Java code with a field named a.b.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-671) SpecificCompiler generates invalid Java
code for invalid identifiers
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-671:
------------------------------
Attachment: AVRO-671.patch
Here's a patch that checks every schema or field name and enum symbol. I modified the constructors for these rather than the parser, so that schemas created programmatically are also checked. I added tests to TestSchema instead of TestSpecificCompiler, since schemas must be created before the compiler operates on them.
This does not validate namespaces, just names.
> SpecificCompiler generates invalid Java code for invalid identifiers
> --------------------------------------------------------------------
>
> Key: AVRO-671
> URL: https://issues.apache.org/jira/browse/AVRO-671
> Project: Avro
> Issue Type: Bug
> Reporter: Ron Bodkin
> Attachments: AVRO-671-r2.patch, AVRO-671-r3.patch, AVRO-671.patch, AVRO-671.patch
>
>
> e.g., if you have a field named "a.b" it will attempt to generate Java code with a field named a.b.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-671) SpecificCompiler generates invalid
Java code for invalid identifiers
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12912649#action_12912649 ]
Doug Cutting commented on AVRO-671:
-----------------------------------
Rather, this should be fixed in Schema.java. A name is not meant to contain such characters:
http://avro.apache.org/docs/current/spec.html#Names
So we should probably add a check into schema construction and throw an exception if a name contains invalid characters.
> SpecificCompiler generates invalid Java code for invalid identifiers
> --------------------------------------------------------------------
>
> Key: AVRO-671
> URL: https://issues.apache.org/jira/browse/AVRO-671
> Project: Avro
> Issue Type: Bug
> Reporter: Ron Bodkin
> Attachments: AVRO-671-r2.patch, AVRO-671.patch
>
>
> e.g., if you have a field named "a.b" it will attempt to generate Java code with a field named a.b.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-671) SpecificCompiler generates invalid Java
code for invalid identifiers
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-671:
------------------------------
Component/s: java
Fix Version/s: 1.4.1
Assignee: Doug Cutting
Unless there are objections, I will commit this soon.
> SpecificCompiler generates invalid Java code for invalid identifiers
> --------------------------------------------------------------------
>
> Key: AVRO-671
> URL: https://issues.apache.org/jira/browse/AVRO-671
> Project: Avro
> Issue Type: Bug
> Components: java
> Reporter: Ron Bodkin
> Assignee: Doug Cutting
> Fix For: 1.4.1
>
> Attachments: AVRO-671-r2.patch, AVRO-671-r3.patch, AVRO-671.patch, AVRO-671.patch
>
>
> e.g., if you have a field named "a.b" it will attempt to generate Java code with a field named a.b.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-671) SpecificCompiler generates invalid
Java code for invalid identifiers
Posted by "Ron Bodkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12912717#action_12912717 ]
Ron Bodkin commented on AVRO-671:
---------------------------------
That sounds fair to me. It does mean that there will be valid JSON documents that can't be translated into Avro (since JSON will accept a lot of other identifier characters). But I can see that it's better to perform the same kind of mangling I proposed in the patch on translation into an Avro format.
> SpecificCompiler generates invalid Java code for invalid identifiers
> --------------------------------------------------------------------
>
> Key: AVRO-671
> URL: https://issues.apache.org/jira/browse/AVRO-671
> Project: Avro
> Issue Type: Bug
> Reporter: Ron Bodkin
> Attachments: AVRO-671-r2.patch, AVRO-671-r3.patch, AVRO-671.patch
>
>
> e.g., if you have a field named "a.b" it will attempt to generate Java code with a field named a.b.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-671) SpecificCompiler generates invalid Java
code for invalid identifiers
Posted by "Ron Bodkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ron Bodkin updated AVRO-671:
----------------------------
Attachment: AVRO-671-r2.patch
Improved patch that handles namespaces and includes tests for those as well.
> SpecificCompiler generates invalid Java code for invalid identifiers
> --------------------------------------------------------------------
>
> Key: AVRO-671
> URL: https://issues.apache.org/jira/browse/AVRO-671
> Project: Avro
> Issue Type: Bug
> Reporter: Ron Bodkin
> Attachments: AVRO-671-r2.patch, AVRO-671.patch
>
>
> e.g., if you have a field named "a.b" it will attempt to generate Java code with a field named a.b.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-671) SpecificCompiler generates invalid
Java code for invalid identifiers
Posted by "Ron Bodkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12911149#action_12911149 ]
Ron Bodkin commented on AVRO-671:
---------------------------------
This patch has a bug - handling of periods is more tricky to allow for proper use of packages but not to generate field names etc. with periods in them. I'm working on a fix for that issue.
> SpecificCompiler generates invalid Java code for invalid identifiers
> --------------------------------------------------------------------
>
> Key: AVRO-671
> URL: https://issues.apache.org/jira/browse/AVRO-671
> Project: Avro
> Issue Type: Bug
> Reporter: Ron Bodkin
> Attachments: AVRO-671.patch
>
>
> e.g., if you have a field named "a.b" it will attempt to generate Java code with a field named a.b.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-671) SpecificCompiler generates invalid Java
code for invalid identifiers
Posted by "Ron Bodkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ron Bodkin updated AVRO-671:
----------------------------
Status: Patch Available (was: Open)
> SpecificCompiler generates invalid Java code for invalid identifiers
> --------------------------------------------------------------------
>
> Key: AVRO-671
> URL: https://issues.apache.org/jira/browse/AVRO-671
> Project: Avro
> Issue Type: Bug
> Reporter: Ron Bodkin
> Attachments: AVRO-671.patch
>
>
> e.g., if you have a field named "a.b" it will attempt to generate Java code with a field named a.b.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-671) SpecificCompiler generates invalid Java
code for invalid identifiers
Posted by "Ron Bodkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ron Bodkin updated AVRO-671:
----------------------------
Attachment: AVRO-671.patch
Patch to fix this.
> SpecificCompiler generates invalid Java code for invalid identifiers
> --------------------------------------------------------------------
>
> Key: AVRO-671
> URL: https://issues.apache.org/jira/browse/AVRO-671
> Project: Avro
> Issue Type: Bug
> Reporter: Ron Bodkin
> Attachments: AVRO-671.patch
>
>
> e.g., if you have a field named "a.b" it will attempt to generate Java code with a field named a.b.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-671) SpecificCompiler generates invalid Java
code for invalid identifiers
Posted by "Ron Bodkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ron Bodkin updated AVRO-671:
----------------------------
Attachment: AVRO-671-r3.patch
Here is a patch that instead throws parse exceptions in the presence of invalid characters.
> SpecificCompiler generates invalid Java code for invalid identifiers
> --------------------------------------------------------------------
>
> Key: AVRO-671
> URL: https://issues.apache.org/jira/browse/AVRO-671
> Project: Avro
> Issue Type: Bug
> Reporter: Ron Bodkin
> Attachments: AVRO-671-r2.patch, AVRO-671-r3.patch, AVRO-671.patch
>
>
> e.g., if you have a field named "a.b" it will attempt to generate Java code with a field named a.b.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-671) SpecificCompiler generates invalid Java
code for invalid identifiers
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-671:
------------------------------
Resolution: Fixed
Fix Version/s: (was: 1.4.1)
1.5.0
Hadoop Flags: [Incompatible change]
Status: Resolved (was: Patch Available)
I committed this, but don't plan to include it in 1.4.1, as it could break applications that use illegal names in schemas.
> SpecificCompiler generates invalid Java code for invalid identifiers
> --------------------------------------------------------------------
>
> Key: AVRO-671
> URL: https://issues.apache.org/jira/browse/AVRO-671
> Project: Avro
> Issue Type: Bug
> Components: java
> Reporter: Ron Bodkin
> Assignee: Doug Cutting
> Fix For: 1.5.0
>
> Attachments: AVRO-671-r2.patch, AVRO-671-r3.patch, AVRO-671.patch, AVRO-671.patch
>
>
> e.g., if you have a field named "a.b" it will attempt to generate Java code with a field named a.b.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.