You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Tom White (JIRA)" <ji...@apache.org> on 2011/08/30 23:39:09 UTC
[jira] [Created] (AVRO-881) Add tool to list data file metadata
Add tool to list data file metadata
-----------------------------------
Key: AVRO-881
URL: https://issues.apache.org/jira/browse/AVRO-881
Project: Avro
Issue Type: New Feature
Components: java
Reporter: Tom White
Assignee: Tom White
It would be useful to be able to query the metadata (e.g. the compression codec being used) from a command line tool.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-881) Add tool to list data file metadata
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated AVRO-881:
---------------------------
Status: Patch Available (was: Open)
> Add tool to list data file metadata
> -----------------------------------
>
> Key: AVRO-881
> URL: https://issues.apache.org/jira/browse/AVRO-881
> Project: Avro
> Issue Type: New Feature
> Components: java
> Reporter: Tom White
> Assignee: Tom White
> Attachments: AVRO-881.patch
>
>
> It would be useful to be able to query the metadata (e.g. the compression codec being used) from a command line tool.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-881) Add tool to list data file metadata
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-881:
------------------------------
Resolution: Fixed
Fix Version/s: 1.6.0
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I updated the test as you suggested and committed this. Thanks, Tom!
> Add tool to list data file metadata
> -----------------------------------
>
> Key: AVRO-881
> URL: https://issues.apache.org/jira/browse/AVRO-881
> Project: Avro
> Issue Type: New Feature
> Components: java
> Reporter: Tom White
> Assignee: Tom White
> Fix For: 1.6.0
>
> Attachments: AVRO-881.patch, AVRO-881.patch
>
>
> It would be useful to be able to query the metadata (e.g. the compression codec being used) from a command line tool.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-881) Add tool to list data file metadata
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13109078#comment-13109078 ]
Tom White commented on AVRO-881:
--------------------------------
Sorry I missed this before. Solution 3 and the patch look fine to me. +1
One minor nit: the test for escaping should use the literal escaped value, rather than calling escapeKey(), so that it tests the escape sequence is as expected.
> Add tool to list data file metadata
> -----------------------------------
>
> Key: AVRO-881
> URL: https://issues.apache.org/jira/browse/AVRO-881
> Project: Avro
> Issue Type: New Feature
> Components: java
> Reporter: Tom White
> Assignee: Tom White
> Attachments: AVRO-881.patch, AVRO-881.patch
>
>
> It would be useful to be able to query the metadata (e.g. the compression codec being used) from a command line tool.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-881) Add tool to list data file metadata
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-881:
------------------------------
Attachment: AVRO-881.patch
Tom, this looks great.
We don't currently prohibit the tab character in metadata keys. So if someone used the tab character in both a metadata key and and value then it would be hard to find the separation between the two. Should we worry about that?
Some possible solutions:
- change the spec & implementations to prohibit tabs in metadata keys
- change the output to be a JSON object (which might be harder for scripts to process)
- escape tabs in keys, replacing them with \t, also escaping CR, NL and backslash too.
Here's a patch that implements that last option.
> Add tool to list data file metadata
> -----------------------------------
>
> Key: AVRO-881
> URL: https://issues.apache.org/jira/browse/AVRO-881
> Project: Avro
> Issue Type: New Feature
> Components: java
> Reporter: Tom White
> Assignee: Tom White
> Attachments: AVRO-881.patch, AVRO-881.patch
>
>
> It would be useful to be able to query the metadata (e.g. the compression codec being used) from a command line tool.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-881) Add tool to list data file metadata
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated AVRO-881:
---------------------------
Attachment: AVRO-881.patch
Example usage:
{noformat}
$ java -jar avro-tools-1.6.0-SNAPSHOT.jar getmeta weather-snappy.avro
avro.codec snappy
avro.schema {"type":"record","name":"Weather","namespace":"test","doc":"A weather reading.","fields":[{"name":"station","type":"string"},{"name":"time","type":"long"},{"name":"temp","type":"int"}]}
$ java -jar avro-tools-1.6.0-SNAPSHOT.jar getmeta weather-snappy.avro --key avro.codec
snappy
{noformat}
> Add tool to list data file metadata
> -----------------------------------
>
> Key: AVRO-881
> URL: https://issues.apache.org/jira/browse/AVRO-881
> Project: Avro
> Issue Type: New Feature
> Components: java
> Reporter: Tom White
> Assignee: Tom White
> Attachments: AVRO-881.patch
>
>
> It would be useful to be able to query the metadata (e.g. the compression codec being used) from a command line tool.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-881) Add tool to list data file metadata
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13109059#comment-13109059 ]
Doug Cutting commented on AVRO-881:
-----------------------------------
I'll commit this tomorrow unless there are objections.
> Add tool to list data file metadata
> -----------------------------------
>
> Key: AVRO-881
> URL: https://issues.apache.org/jira/browse/AVRO-881
> Project: Avro
> Issue Type: New Feature
> Components: java
> Reporter: Tom White
> Assignee: Tom White
> Attachments: AVRO-881.patch, AVRO-881.patch
>
>
> It would be useful to be able to query the metadata (e.g. the compression codec being used) from a command line tool.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira