You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Thiruvalluvan M. G. (JIRA)" <ji...@apache.org> on 2010/12/13 17:48:01 UTC
[jira] Created: (AVRO-711) JSON encoder and decoder for C++
JSON encoder and decoder for C++
--------------------------------
Key: AVRO-711
URL: https://issues.apache.org/jira/browse/AVRO-711
Project: Avro
Issue Type: Improvement
Components: c++
Affects Versions: 1.5.0
Environment: Linux, Cygwin
Reporter: Thiruvalluvan M. G.
Forthcoming patch implements JSON encoder and decoder for C++.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-711) JSON encoder and decoder for C++
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-711:
------------------------------
Fix Version/s: 1.5.0
> JSON encoder and decoder for C++
> --------------------------------
>
> Key: AVRO-711
> URL: https://issues.apache.org/jira/browse/AVRO-711
> Project: Avro
> Issue Type: Improvement
> Components: c++
> Affects Versions: 1.5.0
> Environment: Linux, Cygwin
> Reporter: Thiruvalluvan M. G.
> Assignee: Thiruvalluvan M. G.
> Priority: Blocker
> Fix For: 1.5.0
>
> Attachments: AVRO-711.patch, AVRO-711.patch, AVRO-711.patch, AVRO-711.patch
>
>
> Forthcoming patch implements JSON encoder and decoder for C++.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Issue Comment Edited: (AVRO-711) JSON encoder and decoder
for C++
Posted by "Thiruvalluvan M. G. (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12980233#action_12980233 ]
Thiruvalluvan M. G. edited comment on AVRO-711 at 1/11/11 1:29 PM:
-------------------------------------------------------------------
This is functionally same as the previous patch. But it uses IO streams instead of buffers. This improves performance and paves the way to implement Avro container files.
This patch has been tested on
- Ubuntu 10.04
- RHEL 4.8
- Cygwin
was (Author: thiru_mg):
This is functionally same as the previous patch. But it uses IO streams instead of buffers. This improves performance and paves the way to implement Avro container files.
> JSON encoder and decoder for C++
> --------------------------------
>
> Key: AVRO-711
> URL: https://issues.apache.org/jira/browse/AVRO-711
> Project: Avro
> Issue Type: Improvement
> Components: c++
> Affects Versions: 1.5.0
> Environment: Linux, Cygwin
> Reporter: Thiruvalluvan M. G.
> Assignee: Thiruvalluvan M. G.
> Attachments: AVRO-711.patch, AVRO-711.patch, AVRO-711.patch, AVRO-711.patch
>
>
> Forthcoming patch implements JSON encoder and decoder for C++.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-711) JSON encoder and decoder for C++
Posted by "Thiruvalluvan M. G. (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thiruvalluvan M. G. updated AVRO-711:
-------------------------------------
Attachment: AVRO-711.patch
This is functionally same as the previous patch. But it uses IO streams instead of buffers. This improves performance and paves the way to implement Avro container files.
> JSON encoder and decoder for C++
> --------------------------------
>
> Key: AVRO-711
> URL: https://issues.apache.org/jira/browse/AVRO-711
> Project: Avro
> Issue Type: Improvement
> Components: c++
> Affects Versions: 1.5.0
> Environment: Linux, Cygwin
> Reporter: Thiruvalluvan M. G.
> Assignee: Thiruvalluvan M. G.
> Attachments: AVRO-711.patch, AVRO-711.patch, AVRO-711.patch, AVRO-711.patch
>
>
> Forthcoming patch implements JSON encoder and decoder for C++.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-711) JSON encoder and decoder for C++
Posted by "Thiruvalluvan M. G. (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thiruvalluvan M. G. updated AVRO-711:
-------------------------------------
Attachment: AVRO-711.patch
Slightly better than the previous one. There was a CMake-2.6-specific variation. Removed it.
> JSON encoder and decoder for C++
> --------------------------------
>
> Key: AVRO-711
> URL: https://issues.apache.org/jira/browse/AVRO-711
> Project: Avro
> Issue Type: Improvement
> Components: c++
> Affects Versions: 1.5.0
> Environment: Linux, Cygwin
> Reporter: Thiruvalluvan M. G.
> Assignee: Thiruvalluvan M. G.
> Attachments: AVRO-711.patch, AVRO-711.patch, AVRO-711.patch
>
>
> Forthcoming patch implements JSON encoder and decoder for C++.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-711) JSON encoder and decoder for C++
Posted by "Thiruvalluvan M. G. (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thiruvalluvan M. G. updated AVRO-711:
-------------------------------------
Assignee: Thiruvalluvan M. G.
Status: Patch Available (was: Open)
> JSON encoder and decoder for C++
> --------------------------------
>
> Key: AVRO-711
> URL: https://issues.apache.org/jira/browse/AVRO-711
> Project: Avro
> Issue Type: Improvement
> Components: c++
> Affects Versions: 1.5.0
> Environment: Linux, Cygwin
> Reporter: Thiruvalluvan M. G.
> Assignee: Thiruvalluvan M. G.
> Attachments: AVRO-711.patch
>
>
> Forthcoming patch implements JSON encoder and decoder for C++.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-711) JSON encoder and decoder for C++
Posted by "Scott Banachowski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12972878#action_12972878 ]
Scott Banachowski commented on AVRO-711:
----------------------------------------
Patch works OK for me. Would you like me to check it in now?
> JSON encoder and decoder for C++
> --------------------------------
>
> Key: AVRO-711
> URL: https://issues.apache.org/jira/browse/AVRO-711
> Project: Avro
> Issue Type: Improvement
> Components: c++
> Affects Versions: 1.5.0
> Environment: Linux, Cygwin
> Reporter: Thiruvalluvan M. G.
> Assignee: Thiruvalluvan M. G.
> Attachments: AVRO-711.patch
>
>
> Forthcoming patch implements JSON encoder and decoder for C++.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-711) JSON encoder and decoder for C++
Posted by "Thiruvalluvan M. G. (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thiruvalluvan M. G. updated AVRO-711:
-------------------------------------
Attachment: (was: NOTICE.txt)
> JSON encoder and decoder for C++
> --------------------------------
>
> Key: AVRO-711
> URL: https://issues.apache.org/jira/browse/AVRO-711
> Project: Avro
> Issue Type: Improvement
> Components: c++
> Affects Versions: 1.5.0
> Environment: Linux, Cygwin
> Reporter: Thiruvalluvan M. G.
> Assignee: Thiruvalluvan M. G.
> Attachments: AVRO-711.patch
>
>
> Forthcoming patch implements JSON encoder and decoder for C++.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-711) JSON encoder and decoder for C++
Posted by "Thiruvalluvan M. G. (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thiruvalluvan M. G. updated AVRO-711:
-------------------------------------
Attachment: AVRO-711.patch
This patch implements the JSON encoder and decoder for C++. The design is very similar to that of Java implementation. The patch is incomplete in two respects:
- The decoder does not resolve schema. The Java implementation extends validating decoder to support resolution. I'll submit that as a separate patch.
- The decoder and encoder do not handle recursive schema. They throw when a recursive schema is encountered. Again, I'll address this separately.
Instead of modifying the existing interface, I added two new interfaces Encoder and Decoder in the same lines as Java. So none of the existing applications will have trouble.
The new implementation uses the buffers and validSchema which we already have.
As a side benefit there are binary encoder/decoder and validating encoder/decoder using the new interface.
The new implementation is rigorously tested using the same test cases as that of Java.
This patch may have conflicts with Avro-700. I'll take care of them once Avro-700 is done.
> JSON encoder and decoder for C++
> --------------------------------
>
> Key: AVRO-711
> URL: https://issues.apache.org/jira/browse/AVRO-711
> Project: Avro
> Issue Type: Improvement
> Components: c++
> Affects Versions: 1.5.0
> Environment: Linux, Cygwin
> Reporter: Thiruvalluvan M. G.
> Attachments: AVRO-711.patch
>
>
> Forthcoming patch implements JSON encoder and decoder for C++.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-711) JSON encoder and decoder for C++
Posted by "Thiruvalluvan M. G. (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thiruvalluvan M. G. updated AVRO-711:
-------------------------------------
Attachment: (was: AVRO-711.patch)
> JSON encoder and decoder for C++
> --------------------------------
>
> Key: AVRO-711
> URL: https://issues.apache.org/jira/browse/AVRO-711
> Project: Avro
> Issue Type: Improvement
> Components: c++
> Affects Versions: 1.5.0
> Environment: Linux, Cygwin
> Reporter: Thiruvalluvan M. G.
> Assignee: Thiruvalluvan M. G.
> Attachments: AVRO-711.patch, AVRO-711.patch, AVRO-711.patch
>
>
> Forthcoming patch implements JSON encoder and decoder for C++.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-711) JSON encoder and decoder for C++
Posted by "Thiruvalluvan M. G. (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thiruvalluvan M. G. updated AVRO-711:
-------------------------------------
Attachment: NOTICE.txt
This new patch handles recursive schemas and resolution. The only issue pending is that it does not insert default values for the fields that reader's schema has but writer's schema lacks.
ValidSchema class does not yet have support for default values. We'll address these two together.
> JSON encoder and decoder for C++
> --------------------------------
>
> Key: AVRO-711
> URL: https://issues.apache.org/jira/browse/AVRO-711
> Project: Avro
> Issue Type: Improvement
> Components: c++
> Affects Versions: 1.5.0
> Environment: Linux, Cygwin
> Reporter: Thiruvalluvan M. G.
> Assignee: Thiruvalluvan M. G.
> Attachments: AVRO-711.patch
>
>
> Forthcoming patch implements JSON encoder and decoder for C++.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-711) JSON encoder and decoder for C++
Posted by "Thiruvalluvan M. G. (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thiruvalluvan M. G. updated AVRO-711:
-------------------------------------
Priority: Blocker (was: Major)
> JSON encoder and decoder for C++
> --------------------------------
>
> Key: AVRO-711
> URL: https://issues.apache.org/jira/browse/AVRO-711
> Project: Avro
> Issue Type: Improvement
> Components: c++
> Affects Versions: 1.5.0
> Environment: Linux, Cygwin
> Reporter: Thiruvalluvan M. G.
> Assignee: Thiruvalluvan M. G.
> Priority: Blocker
> Attachments: AVRO-711.patch, AVRO-711.patch, AVRO-711.patch, AVRO-711.patch
>
>
> Forthcoming patch implements JSON encoder and decoder for C++.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (AVRO-711) JSON encoder and decoder for C++
Posted by "Thiruvalluvan M. G. (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thiruvalluvan M. G. updated AVRO-711:
-------------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed revision 1072677 after adjusting fot AVRO-700.
> JSON encoder and decoder for C++
> --------------------------------
>
> Key: AVRO-711
> URL: https://issues.apache.org/jira/browse/AVRO-711
> Project: Avro
> Issue Type: Improvement
> Components: c++
> Affects Versions: 1.5.0
> Environment: Linux, Cygwin
> Reporter: Thiruvalluvan M. G.
> Assignee: Thiruvalluvan M. G.
> Priority: Blocker
> Fix For: 1.5.0
>
> Attachments: AVRO-711.patch, AVRO-711.patch, AVRO-711.patch, AVRO-711.patch
>
>
> Forthcoming patch implements JSON encoder and decoder for C++.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (AVRO-711) JSON encoder and decoder for C++
Posted by "Thiruvalluvan M. G. (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thiruvalluvan M. G. updated AVRO-711:
-------------------------------------
Attachment: AVRO-711.patch
> JSON encoder and decoder for C++
> --------------------------------
>
> Key: AVRO-711
> URL: https://issues.apache.org/jira/browse/AVRO-711
> Project: Avro
> Issue Type: Improvement
> Components: c++
> Affects Versions: 1.5.0
> Environment: Linux, Cygwin
> Reporter: Thiruvalluvan M. G.
> Assignee: Thiruvalluvan M. G.
> Attachments: AVRO-711.patch, AVRO-711.patch
>
>
> Forthcoming patch implements JSON encoder and decoder for C++.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (AVRO-711) JSON encoder and decoder
for C++
Posted by "Thiruvalluvan M. G. (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12970896#action_12970896 ]
Thiruvalluvan M. G. edited comment on AVRO-711 at 12/13/10 12:04 PM:
---------------------------------------------------------------------
This patch implements the JSON encoder and decoder for C++. The design is very similar to that of Java implementation. The patch is incomplete in two respects:
- The decoder does not resolve schema. The Java implementation extends validating decoder to support resolution. I'll submit that as a separate patch.
- The decoder and encoder do not handle recursive schema. They throw when a recursive schema is encountered. Again, I'll address this separately.
Instead of modifying the existing interface, I added two new interfaces Encoder and Decoder in the same lines as Java. So none of the existing applications will have trouble.
The new implementation uses the buffers and validSchema which we already have.
As a side benefit there are binary encoder/decoder and validating encoder/decoder using the new interface.
The new implementation is rigorously tested using the same test cases as that of Java.
This patch may have conflicts with AVRO-700. I'll take care of them once AVRO-700 is done.
was (Author: thiru_mg):
This patch implements the JSON encoder and decoder for C++. The design is very similar to that of Java implementation. The patch is incomplete in two respects:
- The decoder does not resolve schema. The Java implementation extends validating decoder to support resolution. I'll submit that as a separate patch.
- The decoder and encoder do not handle recursive schema. They throw when a recursive schema is encountered. Again, I'll address this separately.
Instead of modifying the existing interface, I added two new interfaces Encoder and Decoder in the same lines as Java. So none of the existing applications will have trouble.
The new implementation uses the buffers and validSchema which we already have.
As a side benefit there are binary encoder/decoder and validating encoder/decoder using the new interface.
The new implementation is rigorously tested using the same test cases as that of Java.
This patch may have conflicts with Avro-700. I'll take care of them once Avro-700 is done.
> JSON encoder and decoder for C++
> --------------------------------
>
> Key: AVRO-711
> URL: https://issues.apache.org/jira/browse/AVRO-711
> Project: Avro
> Issue Type: Improvement
> Components: c++
> Affects Versions: 1.5.0
> Environment: Linux, Cygwin
> Reporter: Thiruvalluvan M. G.
> Assignee: Thiruvalluvan M. G.
> Attachments: AVRO-711.patch
>
>
> Forthcoming patch implements JSON encoder and decoder for C++.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-711) JSON encoder and decoder for C++
Posted by "Thiruvalluvan M. G. (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thiruvalluvan M. G. updated AVRO-711:
-------------------------------------
Attachment: AVRO-711.patch
Same as the previous patch. But this is tested on the following:
- RHEL 4.7 with CMake 2.6, Boost 1.41.0
- Ubuntu 10.04 with CMake 2.8.0, Boost 1.38.0
- Cygwin 1.7 with CMake 2.8.2, Boost 1.43.0
> JSON encoder and decoder for C++
> --------------------------------
>
> Key: AVRO-711
> URL: https://issues.apache.org/jira/browse/AVRO-711
> Project: Avro
> Issue Type: Improvement
> Components: c++
> Affects Versions: 1.5.0
> Environment: Linux, Cygwin
> Reporter: Thiruvalluvan M. G.
> Assignee: Thiruvalluvan M. G.
> Attachments: AVRO-711.patch, AVRO-711.patch, AVRO-711.patch
>
>
> Forthcoming patch implements JSON encoder and decoder for C++.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.