You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Harsh J Chouraria (JIRA)" <ji...@apache.org> on 2010/08/28 07:14:53 UTC
[jira] Created: (AVRO-642) Prettify the json dumped during
exceptions while validating a writer/reader schema
Prettify the json dumped during exceptions while validating a writer/reader schema
----------------------------------------------------------------------------------
Key: AVRO-642
URL: https://issues.apache.org/jira/browse/AVRO-642
Project: Avro
Issue Type: Improvement
Components: java, python
Affects Versions: 1.4.0
Environment: Avro-Trunk
Reporter: Harsh J Chouraria
Assignee: Harsh J Chouraria
Priority: Minor
Fix For: 1.4.0
Attachments: avro.java.schema.pretty.r1.diff, avro.py.schema.pretty.r1.diff
Right now the exception that's thrown when a reader and written schema don't match (while reading a file), looks like this:
{noformat}
Found {"type":"record","name":"Test","fields":[{"name":"stringField","type":"string"},{"name":"longField","type":"long"}]}, expecting {"type":"record","name":"Test1","fields":[{"name":"stringField","type":"string"},{"name":"longField","type":"long"}]}
{noformat}
While this is perfectly okay for small schema with a few fields, it gets hard to debug for complex and large ones.
The attached patch changes the exception message to look like:
{noformat}
Found {
"type" : "record",
"name" : "Test",
"fields" : [ {
"name" : "stringField",
"type" : "string"
}, {
"name" : "longField",
"type" : "long"
} ]
}, expecting {
"type" : "record",
"name" : "Test1",
"fields" : [ {
"name" : "stringField",
"type" : "string"
}, {
"name" : "longField",
"type" : "long"
} ]
}
{noformat}
P.s. Ideally a diff-like message would help too, but its mileage may vary. I guess with the pretty output printed, a user may easily do a diff (since they now have newlines), so us providing this is not _really_ needed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (AVRO-642) Prettify the json dumped during
exceptions while validating a writer/reader schema
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting resolved AVRO-642.
-------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
I committed this. Thanks, Harsh.
> Prettify the json dumped during exceptions while validating a writer/reader schema
> ----------------------------------------------------------------------------------
>
> Key: AVRO-642
> URL: https://issues.apache.org/jira/browse/AVRO-642
> Project: Avro
> Issue Type: Improvement
> Components: java, python
> Affects Versions: 1.3.3
> Environment: Avro-Trunk
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Minor
> Fix For: 1.5.0
>
> Attachments: avro.java.schema.pretty.r1.diff, avro.py.schema.pretty.r1.diff, avro.schema.pretty.r2.diff, avro.schema.pretty.r3.diff, avro.schema.pretty.r4.diff, avro.schema.pretty.r5.diff
>
> Original Estimate: 0.02h
> Remaining Estimate: 0.02h
>
> Right now the exception that's thrown when a reader and written schema don't match (while reading a file), looks like this:
> {noformat}
> Found {"type":"record","name":"Test","fields":[{"name":"stringField","type":"string"},{"name":"longField","type":"long"}]}, expecting {"type":"record","name":"Test1","fields":[{"name":"stringField","type":"string"},{"name":"longField","type":"long"}]}
> {noformat}
> While this is perfectly okay for small schema with a few fields, it gets hard to debug for complex and large ones.
> The attached patch changes the exception message to look like:
> {noformat}
> Found {
> "type" : "record",
> "name" : "Test",
> "fields" : [ {
> "name" : "stringField",
> "type" : "string"
> }, {
> "name" : "longField",
> "type" : "long"
> } ]
> }, expecting {
> "type" : "record",
> "name" : "Test1",
> "fields" : [ {
> "name" : "stringField",
> "type" : "string"
> }, {
> "name" : "longField",
> "type" : "long"
> } ]
> }
> {noformat}
> P.s. Ideally a diff-like message would help too, but its mileage may vary. I guess with the pretty output printed, a user may easily do a diff (since they now have newlines), so us providing this is not _really_ needed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-642) Prettify the json dumped during
exceptions while validating a writer/reader schema
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12904265#action_12904265 ]
Doug Cutting commented on AVRO-642:
-----------------------------------
This causes Python unit tests to fail for me. Also, can you please post this as a single patch file? Thanks!
> Prettify the json dumped during exceptions while validating a writer/reader schema
> ----------------------------------------------------------------------------------
>
> Key: AVRO-642
> URL: https://issues.apache.org/jira/browse/AVRO-642
> Project: Avro
> Issue Type: Improvement
> Components: java, python
> Affects Versions: 1.3.3
> Environment: Avro-Trunk
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Minor
> Fix For: 1.4.1
>
> Attachments: avro.java.schema.pretty.r1.diff, avro.py.schema.pretty.r1.diff
>
> Original Estimate: 0.02h
> Remaining Estimate: 0.02h
>
> Right now the exception that's thrown when a reader and written schema don't match (while reading a file), looks like this:
> {noformat}
> Found {"type":"record","name":"Test","fields":[{"name":"stringField","type":"string"},{"name":"longField","type":"long"}]}, expecting {"type":"record","name":"Test1","fields":[{"name":"stringField","type":"string"},{"name":"longField","type":"long"}]}
> {noformat}
> While this is perfectly okay for small schema with a few fields, it gets hard to debug for complex and large ones.
> The attached patch changes the exception message to look like:
> {noformat}
> Found {
> "type" : "record",
> "name" : "Test",
> "fields" : [ {
> "name" : "stringField",
> "type" : "string"
> }, {
> "name" : "longField",
> "type" : "long"
> } ]
> }, expecting {
> "type" : "record",
> "name" : "Test1",
> "fields" : [ {
> "name" : "stringField",
> "type" : "string"
> }, {
> "name" : "longField",
> "type" : "long"
> } ]
> }
> {noformat}
> P.s. Ideally a diff-like message would help too, but its mileage may vary. I guess with the pretty output printed, a user may easily do a diff (since they now have newlines), so us providing this is not _really_ needed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-642) Prettify the json dumped during
exceptions while validating a writer/reader schema
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J Chouraria updated AVRO-642:
-----------------------------------
Attachment: avro.schema.pretty.r5.diff
Forgot java on the other one..
> Prettify the json dumped during exceptions while validating a writer/reader schema
> ----------------------------------------------------------------------------------
>
> Key: AVRO-642
> URL: https://issues.apache.org/jira/browse/AVRO-642
> Project: Avro
> Issue Type: Improvement
> Components: java, python
> Affects Versions: 1.3.3
> Environment: Avro-Trunk
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Minor
> Fix For: 1.5.0
>
> Attachments: avro.java.schema.pretty.r1.diff, avro.py.schema.pretty.r1.diff, avro.schema.pretty.r2.diff, avro.schema.pretty.r3.diff, avro.schema.pretty.r4.diff, avro.schema.pretty.r5.diff
>
> Original Estimate: 0.02h
> Remaining Estimate: 0.02h
>
> Right now the exception that's thrown when a reader and written schema don't match (while reading a file), looks like this:
> {noformat}
> Found {"type":"record","name":"Test","fields":[{"name":"stringField","type":"string"},{"name":"longField","type":"long"}]}, expecting {"type":"record","name":"Test1","fields":[{"name":"stringField","type":"string"},{"name":"longField","type":"long"}]}
> {noformat}
> While this is perfectly okay for small schema with a few fields, it gets hard to debug for complex and large ones.
> The attached patch changes the exception message to look like:
> {noformat}
> Found {
> "type" : "record",
> "name" : "Test",
> "fields" : [ {
> "name" : "stringField",
> "type" : "string"
> }, {
> "name" : "longField",
> "type" : "long"
> } ]
> }, expecting {
> "type" : "record",
> "name" : "Test1",
> "fields" : [ {
> "name" : "stringField",
> "type" : "string"
> }, {
> "name" : "longField",
> "type" : "long"
> } ]
> }
> {noformat}
> P.s. Ideally a diff-like message would help too, but its mileage may vary. I guess with the pretty output printed, a user may easily do a diff (since they now have newlines), so us providing this is not _really_ needed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-642) Prettify the json dumped during
exceptions while validating a writer/reader schema
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12924711#action_12924711 ]
Doug Cutting commented on AVRO-642:
-----------------------------------
Here's the python error I see for this.
{code}
ERROR: test_type_exception (test_io.TestIO)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/cutting/src/avro/trunk/lang/py/build/test/test_io.py", line 334, in test_type_exception
self.assertRaises(io.AvroTypeException, write_datum, datum_to_write, writers_schema)
File "/usr/lib/python2.6/unittest.py", line 336, in failUnlessRaises
callableObj(*args, **kwargs)
File "/home/cutting/src/avro/trunk/lang/py/build/test/test_io.py", line 119, in write_datum
datum_writer.write(datum, encoder)
File "/home/cutting/src/avro/trunk/lang/py/build/src/avro/io.py", line 750, in write
raise AvroTypeException(self.writers_schema, datum)
File "/home/cutting/src/avro/trunk/lang/py/build/src/avro/io.py", line 77, in __init__
pretty_expected = json.dumps(json.loads(str(expected_schema)), indent=2)
NameError: global name 'json' is not defined
{code}
> Prettify the json dumped during exceptions while validating a writer/reader schema
> ----------------------------------------------------------------------------------
>
> Key: AVRO-642
> URL: https://issues.apache.org/jira/browse/AVRO-642
> Project: Avro
> Issue Type: Improvement
> Components: java, python
> Affects Versions: 1.3.3
> Environment: Avro-Trunk
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Minor
> Fix For: 1.5.0
>
> Attachments: avro.java.schema.pretty.r1.diff, avro.py.schema.pretty.r1.diff, avro.schema.pretty.r2.diff
>
> Original Estimate: 0.02h
> Remaining Estimate: 0.02h
>
> Right now the exception that's thrown when a reader and written schema don't match (while reading a file), looks like this:
> {noformat}
> Found {"type":"record","name":"Test","fields":[{"name":"stringField","type":"string"},{"name":"longField","type":"long"}]}, expecting {"type":"record","name":"Test1","fields":[{"name":"stringField","type":"string"},{"name":"longField","type":"long"}]}
> {noformat}
> While this is perfectly okay for small schema with a few fields, it gets hard to debug for complex and large ones.
> The attached patch changes the exception message to look like:
> {noformat}
> Found {
> "type" : "record",
> "name" : "Test",
> "fields" : [ {
> "name" : "stringField",
> "type" : "string"
> }, {
> "name" : "longField",
> "type" : "long"
> } ]
> }, expecting {
> "type" : "record",
> "name" : "Test1",
> "fields" : [ {
> "name" : "stringField",
> "type" : "string"
> }, {
> "name" : "longField",
> "type" : "long"
> } ]
> }
> {noformat}
> P.s. Ideally a diff-like message would help too, but its mileage may vary. I guess with the pretty output printed, a user may easily do a diff (since they now have newlines), so us providing this is not _really_ needed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-642) Prettify the json dumped during
exceptions while validating a writer/reader schema
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J Chouraria updated AVRO-642:
-----------------------------------
Attachment: avro.schema.pretty.r2.diff
Updated patch to include both languages.
Python tests: I see all tests passing in test_io.py. Could you tell me what test was failing with the earlier patch Doug? I'm only modifying fail_msg inside a python exception, which should never be really relied upon and utilized to retrieve stuff back, correct?
> Prettify the json dumped during exceptions while validating a writer/reader schema
> ----------------------------------------------------------------------------------
>
> Key: AVRO-642
> URL: https://issues.apache.org/jira/browse/AVRO-642
> Project: Avro
> Issue Type: Improvement
> Components: java, python
> Affects Versions: 1.3.3
> Environment: Avro-Trunk
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Minor
> Fix For: 1.5.0
>
> Attachments: avro.java.schema.pretty.r1.diff, avro.py.schema.pretty.r1.diff, avro.schema.pretty.r2.diff
>
> Original Estimate: 0.02h
> Remaining Estimate: 0.02h
>
> Right now the exception that's thrown when a reader and written schema don't match (while reading a file), looks like this:
> {noformat}
> Found {"type":"record","name":"Test","fields":[{"name":"stringField","type":"string"},{"name":"longField","type":"long"}]}, expecting {"type":"record","name":"Test1","fields":[{"name":"stringField","type":"string"},{"name":"longField","type":"long"}]}
> {noformat}
> While this is perfectly okay for small schema with a few fields, it gets hard to debug for complex and large ones.
> The attached patch changes the exception message to look like:
> {noformat}
> Found {
> "type" : "record",
> "name" : "Test",
> "fields" : [ {
> "name" : "stringField",
> "type" : "string"
> }, {
> "name" : "longField",
> "type" : "long"
> } ]
> }, expecting {
> "type" : "record",
> "name" : "Test1",
> "fields" : [ {
> "name" : "stringField",
> "type" : "string"
> }, {
> "name" : "longField",
> "type" : "long"
> } ]
> }
> {noformat}
> P.s. Ideally a diff-like message would help too, but its mileage may vary. I guess with the pretty output printed, a user may easily do a diff (since they now have newlines), so us providing this is not _really_ needed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-642) Prettify the json dumped during
exceptions while validating a writer/reader schema
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J Chouraria updated AVRO-642:
-----------------------------------
Attachment: avro.java.schema.pretty.r1.diff
avro.py.schema.pretty.r1.diff
Patches for Python and Java that prettify the schema output.
> Prettify the json dumped during exceptions while validating a writer/reader schema
> ----------------------------------------------------------------------------------
>
> Key: AVRO-642
> URL: https://issues.apache.org/jira/browse/AVRO-642
> Project: Avro
> Issue Type: Improvement
> Components: java, python
> Affects Versions: 1.4.0
> Environment: Avro-Trunk
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Minor
> Fix For: 1.4.0
>
> Attachments: avro.java.schema.pretty.r1.diff, avro.py.schema.pretty.r1.diff
>
> Original Estimate: 0.02h
> Remaining Estimate: 0.02h
>
> Right now the exception that's thrown when a reader and written schema don't match (while reading a file), looks like this:
> {noformat}
> Found {"type":"record","name":"Test","fields":[{"name":"stringField","type":"string"},{"name":"longField","type":"long"}]}, expecting {"type":"record","name":"Test1","fields":[{"name":"stringField","type":"string"},{"name":"longField","type":"long"}]}
> {noformat}
> While this is perfectly okay for small schema with a few fields, it gets hard to debug for complex and large ones.
> The attached patch changes the exception message to look like:
> {noformat}
> Found {
> "type" : "record",
> "name" : "Test",
> "fields" : [ {
> "name" : "stringField",
> "type" : "string"
> }, {
> "name" : "longField",
> "type" : "long"
> } ]
> }, expecting {
> "type" : "record",
> "name" : "Test1",
> "fields" : [ {
> "name" : "stringField",
> "type" : "string"
> }, {
> "name" : "longField",
> "type" : "long"
> } ]
> }
> {noformat}
> P.s. Ideally a diff-like message would help too, but its mileage may vary. I guess with the pretty output printed, a user may easily do a diff (since they now have newlines), so us providing this is not _really_ needed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-642) Prettify the json dumped during
exceptions while validating a writer/reader schema
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12925285#action_12925285 ]
Harsh J Chouraria commented on AVRO-642:
----------------------------------------
Odd, I seem to get a success message (or no message/exception) for the same command.
{code}
hadoop@hadoop:~/home/Avro/lang/py$ ant clean test
(...)
[py-test] Value to Skip: -8193
[py-test] Read Value: 6253
[py-test]
[py-test]
[py-test] TEST TYPE EXCEPTION
[py-test] ===================
[py-test]
[py-test]
[py-test] TEST UNKNOWN SYMBOL
[py-test] ===================
[py-test]
[py-test]
[py-test] TEST VALIDATE
[py-test] =============
[py-test]
[py-test] Schema: "null"
[py-test] Datum: None
(...)
BUILD SUCCESSFUL
Total time: 1 second
{code}
Could it be the stringify operation [str()] on the schema that's going wrong?
> Prettify the json dumped during exceptions while validating a writer/reader schema
> ----------------------------------------------------------------------------------
>
> Key: AVRO-642
> URL: https://issues.apache.org/jira/browse/AVRO-642
> Project: Avro
> Issue Type: Improvement
> Components: java, python
> Affects Versions: 1.3.3
> Environment: Avro-Trunk
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Minor
> Fix For: 1.5.0
>
> Attachments: avro.java.schema.pretty.r1.diff, avro.py.schema.pretty.r1.diff, avro.schema.pretty.r2.diff, avro.schema.pretty.r3.diff
>
> Original Estimate: 0.02h
> Remaining Estimate: 0.02h
>
> Right now the exception that's thrown when a reader and written schema don't match (while reading a file), looks like this:
> {noformat}
> Found {"type":"record","name":"Test","fields":[{"name":"stringField","type":"string"},{"name":"longField","type":"long"}]}, expecting {"type":"record","name":"Test1","fields":[{"name":"stringField","type":"string"},{"name":"longField","type":"long"}]}
> {noformat}
> While this is perfectly okay for small schema with a few fields, it gets hard to debug for complex and large ones.
> The attached patch changes the exception message to look like:
> {noformat}
> Found {
> "type" : "record",
> "name" : "Test",
> "fields" : [ {
> "name" : "stringField",
> "type" : "string"
> }, {
> "name" : "longField",
> "type" : "long"
> } ]
> }, expecting {
> "type" : "record",
> "name" : "Test1",
> "fields" : [ {
> "name" : "stringField",
> "type" : "string"
> }, {
> "name" : "longField",
> "type" : "long"
> } ]
> }
> {noformat}
> P.s. Ideally a diff-like message would help too, but its mileage may vary. I guess with the pretty output printed, a user may easily do a diff (since they now have newlines), so us providing this is not _really_ needed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-642) Prettify the json dumped during
exceptions while validating a writer/reader schema
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12925120#action_12925120 ]
Doug Cutting commented on AVRO-642:
-----------------------------------
I now see the following error when I run 'cd lang/py; ant clean test'.
{code}
[py-test] ======================================================================
[py-test] ERROR: test_type_exception (test_io.TestIO)
[py-test] ----------------------------------------------------------------------
[py-test] Traceback (most recent call last):
[py-test] File "/home/cutting/src/avro/trunk/lang/py/build/test/test_io.py", line 334, in test_type_exception
[py-test] self.assertRaises(io.AvroTypeException, write_datum, datum_to_write, writers_schema)
[py-test] File "/usr/lib/python2.6/unittest.py", line 336, in failUnlessRaises
[py-test] callableObj(*args, **kwargs)
[py-test] File "/home/cutting/src/avro/trunk/lang/py/build/test/test_io.py", line 119, in write_datum
[py-test] datum_writer.write(datum, encoder)
[py-test] File "/home/cutting/src/avro/trunk/lang/py/build/src/avro/io.py", line 755, in write
[py-test] raise AvroTypeException(self.writers_schema, datum)
[py-test] File "/home/cutting/src/avro/trunk/lang/py/build/src/avro/io.py", line 83, in __init__
[py-test] pretty_datum = json.dumps(json.loads(str(datum)), indent=2)
[py-test] File "/usr/lib/python2.6/json/__init__.py", line 307, in loads
[py-test] return _default_decoder.decode(s)
[py-test] File "/usr/lib/python2.6/json/decoder.py", line 319, in decode
[py-test] obj, end = self.raw_decode(s, idx=_w(s, 0).end())
[py-test] File "/usr/lib/python2.6/json/decoder.py", line 336, in raw_decode
[py-test] obj, end = self._scanner.iterscan(s, **kw).next()
[py-test] File "/usr/lib/python2.6/json/scanner.py", line 55, in iterscan
[py-test] rval, next_pos = action(m, context)
[py-test] File "/usr/lib/python2.6/json/decoder.py", line 171, in JSONObject
[py-test] raise ValueError(errmsg("Expecting property name", s, end))
[py-test] ValueError: Expecting property name: line 1 column 1 (char 1)
{code}
> Prettify the json dumped during exceptions while validating a writer/reader schema
> ----------------------------------------------------------------------------------
>
> Key: AVRO-642
> URL: https://issues.apache.org/jira/browse/AVRO-642
> Project: Avro
> Issue Type: Improvement
> Components: java, python
> Affects Versions: 1.3.3
> Environment: Avro-Trunk
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Minor
> Fix For: 1.5.0
>
> Attachments: avro.java.schema.pretty.r1.diff, avro.py.schema.pretty.r1.diff, avro.schema.pretty.r2.diff, avro.schema.pretty.r3.diff
>
> Original Estimate: 0.02h
> Remaining Estimate: 0.02h
>
> Right now the exception that's thrown when a reader and written schema don't match (while reading a file), looks like this:
> {noformat}
> Found {"type":"record","name":"Test","fields":[{"name":"stringField","type":"string"},{"name":"longField","type":"long"}]}, expecting {"type":"record","name":"Test1","fields":[{"name":"stringField","type":"string"},{"name":"longField","type":"long"}]}
> {noformat}
> While this is perfectly okay for small schema with a few fields, it gets hard to debug for complex and large ones.
> The attached patch changes the exception message to look like:
> {noformat}
> Found {
> "type" : "record",
> "name" : "Test",
> "fields" : [ {
> "name" : "stringField",
> "type" : "string"
> }, {
> "name" : "longField",
> "type" : "long"
> } ]
> }, expecting {
> "type" : "record",
> "name" : "Test1",
> "fields" : [ {
> "name" : "stringField",
> "type" : "string"
> }, {
> "name" : "longField",
> "type" : "long"
> } ]
> }
> {noformat}
> P.s. Ideally a diff-like message would help too, but its mileage may vary. I guess with the pretty output printed, a user may easily do a diff (since they now have newlines), so us providing this is not _really_ needed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-642) Prettify the json dumped during
exceptions while validating a writer/reader schema
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J Chouraria updated AVRO-642:
-----------------------------------
Fix Version/s: 1.4.1
(was: 1.4.0)
Affects Version/s: 1.3.3
(was: 1.4.0)
> Prettify the json dumped during exceptions while validating a writer/reader schema
> ----------------------------------------------------------------------------------
>
> Key: AVRO-642
> URL: https://issues.apache.org/jira/browse/AVRO-642
> Project: Avro
> Issue Type: Improvement
> Components: java, python
> Affects Versions: 1.3.3
> Environment: Avro-Trunk
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Minor
> Fix For: 1.4.1
>
> Attachments: avro.java.schema.pretty.r1.diff, avro.py.schema.pretty.r1.diff
>
> Original Estimate: 0.02h
> Remaining Estimate: 0.02h
>
> Right now the exception that's thrown when a reader and written schema don't match (while reading a file), looks like this:
> {noformat}
> Found {"type":"record","name":"Test","fields":[{"name":"stringField","type":"string"},{"name":"longField","type":"long"}]}, expecting {"type":"record","name":"Test1","fields":[{"name":"stringField","type":"string"},{"name":"longField","type":"long"}]}
> {noformat}
> While this is perfectly okay for small schema with a few fields, it gets hard to debug for complex and large ones.
> The attached patch changes the exception message to look like:
> {noformat}
> Found {
> "type" : "record",
> "name" : "Test",
> "fields" : [ {
> "name" : "stringField",
> "type" : "string"
> }, {
> "name" : "longField",
> "type" : "long"
> } ]
> }, expecting {
> "type" : "record",
> "name" : "Test1",
> "fields" : [ {
> "name" : "stringField",
> "type" : "string"
> }, {
> "name" : "longField",
> "type" : "long"
> } ]
> }
> {noformat}
> P.s. Ideally a diff-like message would help too, but its mileage may vary. I guess with the pretty output printed, a user may easily do a diff (since they now have newlines), so us providing this is not _really_ needed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-642) Prettify the json dumped during
exceptions while validating a writer/reader schema
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J Chouraria updated AVRO-642:
-----------------------------------
Attachment: avro.schema.pretty.r4.diff
My bad. Fixed in this patch. Could you check if the tests are passing now?
The test_io.py seems to pass for me (earlier it was not picking up the edited avro module, sorry).
> Prettify the json dumped during exceptions while validating a writer/reader schema
> ----------------------------------------------------------------------------------
>
> Key: AVRO-642
> URL: https://issues.apache.org/jira/browse/AVRO-642
> Project: Avro
> Issue Type: Improvement
> Components: java, python
> Affects Versions: 1.3.3
> Environment: Avro-Trunk
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Minor
> Fix For: 1.5.0
>
> Attachments: avro.java.schema.pretty.r1.diff, avro.py.schema.pretty.r1.diff, avro.schema.pretty.r2.diff, avro.schema.pretty.r3.diff, avro.schema.pretty.r4.diff
>
> Original Estimate: 0.02h
> Remaining Estimate: 0.02h
>
> Right now the exception that's thrown when a reader and written schema don't match (while reading a file), looks like this:
> {noformat}
> Found {"type":"record","name":"Test","fields":[{"name":"stringField","type":"string"},{"name":"longField","type":"long"}]}, expecting {"type":"record","name":"Test1","fields":[{"name":"stringField","type":"string"},{"name":"longField","type":"long"}]}
> {noformat}
> While this is perfectly okay for small schema with a few fields, it gets hard to debug for complex and large ones.
> The attached patch changes the exception message to look like:
> {noformat}
> Found {
> "type" : "record",
> "name" : "Test",
> "fields" : [ {
> "name" : "stringField",
> "type" : "string"
> }, {
> "name" : "longField",
> "type" : "long"
> } ]
> }, expecting {
> "type" : "record",
> "name" : "Test1",
> "fields" : [ {
> "name" : "stringField",
> "type" : "string"
> }, {
> "name" : "longField",
> "type" : "long"
> } ]
> }
> {noformat}
> P.s. Ideally a diff-like message would help too, but its mileage may vary. I guess with the pretty output printed, a user may easily do a diff (since they now have newlines), so us providing this is not _really_ needed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-642) Prettify the json dumped during
exceptions while validating a writer/reader schema
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J Chouraria updated AVRO-642:
-----------------------------------
Attachment: avro.schema.pretty.r3.diff
Oh, oops. I didn't check the imports.
Fixed in this patch.
I ran: ant test, ant interop-data-generate, ant interop-data-test. All seem to pass.
> Prettify the json dumped during exceptions while validating a writer/reader schema
> ----------------------------------------------------------------------------------
>
> Key: AVRO-642
> URL: https://issues.apache.org/jira/browse/AVRO-642
> Project: Avro
> Issue Type: Improvement
> Components: java, python
> Affects Versions: 1.3.3
> Environment: Avro-Trunk
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Minor
> Fix For: 1.5.0
>
> Attachments: avro.java.schema.pretty.r1.diff, avro.py.schema.pretty.r1.diff, avro.schema.pretty.r2.diff, avro.schema.pretty.r3.diff
>
> Original Estimate: 0.02h
> Remaining Estimate: 0.02h
>
> Right now the exception that's thrown when a reader and written schema don't match (while reading a file), looks like this:
> {noformat}
> Found {"type":"record","name":"Test","fields":[{"name":"stringField","type":"string"},{"name":"longField","type":"long"}]}, expecting {"type":"record","name":"Test1","fields":[{"name":"stringField","type":"string"},{"name":"longField","type":"long"}]}
> {noformat}
> While this is perfectly okay for small schema with a few fields, it gets hard to debug for complex and large ones.
> The attached patch changes the exception message to look like:
> {noformat}
> Found {
> "type" : "record",
> "name" : "Test",
> "fields" : [ {
> "name" : "stringField",
> "type" : "string"
> }, {
> "name" : "longField",
> "type" : "long"
> } ]
> }, expecting {
> "type" : "record",
> "name" : "Test1",
> "fields" : [ {
> "name" : "stringField",
> "type" : "string"
> }, {
> "name" : "longField",
> "type" : "long"
> } ]
> }
> {noformat}
> P.s. Ideally a diff-like message would help too, but its mileage may vary. I guess with the pretty output printed, a user may easily do a diff (since they now have newlines), so us providing this is not _really_ needed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.