You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Matt Massie (JIRA)" <ji...@apache.org> on 2009/11/20 08:55:40 UTC
[jira] Created: (AVRO-230) Create a shared schema test directory
structure
Create a shared schema test directory structure
-----------------------------------------------
Key: AVRO-230
URL: https://issues.apache.org/jira/browse/AVRO-230
Project: Avro
Issue Type: New Feature
Components: c, c++, java, python
Reporter: Matt Massie
Assignee: Matt Massie
Fix For: 1.3.0
This is an example of my proposed directory structure:
* invalid_schemas/
** broken.json
** wrong.json
* valid_data/
** foo_test/
*** schema.json
*** json_data/
**** valid_json_test_data.json
**** more_valid_json_test_data.json
*** binary_data/
**** valid_binary_test_data.bin
**** more_test_data.bin
** bar_test/
*** schema.json
*** json_data/
**** ...
* invalid_data/
** baz_test/
*** schema.json
*** json_data/
**** ...
*** binary_data/
**** ...
This structure supports positive and negative tests for avro schemas, json data and binary data.
* The "invalid_schema" directory holds a number of invalid schemas that should fail to parse.
* The "valid_data" directory has a number of self-contained tests in separate directories. Each test directory is required to have a "schema.json" file that valid avro schema. The "json_data" and "binary_data" directories are optional for each test.
* The "invalid_data" directory has the same rules as the "valid_data" directory. The data files should fail during tests (negative testing).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-230) Create a shared schema test directory
structure
Posted by "Matt Massie (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12780714#action_12780714 ]
Matt Massie commented on AVRO-230:
----------------------------------
bq. I'd vote for a "valid_schemas" directory too.
Jeff: By definition, all schema in the "valid_data" directory must be valid and the "json_data" and "binary_data" directories are optional. So it would be easy to test an array of schemas without much hassle.
e.g.
valid_data
* test_foo
** schema.json
* test_bar
** schema.json
* test_baz
** schema.json
would be all you need to test three schema without any data tests.
bq. We should probably do the same thing for valid/invalid protocols.
Jeff: +1 on your and Doug's suggestion to also include valid requests and responses
bq. In valid_data/binary_data, should each file contain just a single serialized object with no headers, etc, or should it be a data file?
Doug: I think you're correct in have a separate directory for single objects and objects inside a container. This would allow people writing new language bindings to test objects eoncoding/decoding without needing to implement any containers.
bq. To support validation, we could store a random seed in data file's metadata. Then a validator can read the file while generating random objects from the seed and schema and check that the two match.
Doug: +1 I really like this idea. Couldn't we just (1) seed with a fixed number, (2) encode schema with random data and then (3) save away the next random number? The implementation validating the schema would then (1) seed with a fixed number, (2) decode the data and (3) generate a random number and compare to the encoders stored value?
Or did I just repeat your recommendation using different words?
> Create a shared schema test directory structure
> -----------------------------------------------
>
> Key: AVRO-230
> URL: https://issues.apache.org/jira/browse/AVRO-230
> Project: Avro
> Issue Type: New Feature
> Components: c, c++, java, python
> Reporter: Matt Massie
> Assignee: Matt Massie
> Fix For: 1.3.0
>
>
> This is an example of my proposed directory structure:
> * invalid_schemas/
> ** broken.json
> ** wrong.json
> * valid_data/
> ** foo_test/
> *** schema.json
> *** json_data/
> **** valid_json_test_data.json
> **** more_valid_json_test_data.json
> *** binary_data/
> **** valid_binary_test_data.bin
> **** more_test_data.bin
> ** bar_test/
> *** schema.json
> *** json_data/
> **** ...
> * invalid_data/
> ** baz_test/
> *** schema.json
> *** json_data/
> **** ...
> *** binary_data/
> **** ...
> This structure supports positive and negative tests for avro schemas, json data and binary data.
> * The "invalid_schema" directory holds a number of invalid schemas that should fail to parse.
> * The "valid_data" directory has a number of self-contained tests in separate directories. Each test directory is required to have a "schema.json" file that valid avro schema. The "json_data" and "binary_data" directories are optional for each test.
> * The "invalid_data" directory has the same rules as the "valid_data" directory. The data files should fail during tests (negative testing).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-230) Create a shared schema test directory
structure
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12780693#action_12780693 ]
Doug Cutting commented on AVRO-230:
-----------------------------------
> I'd vote for a "valid_schemas" directory too.
The valid_data directory has valid schemas. Is that enough?
In valid_data/binary_data, should each file contain just a single serialized object with no headers, etc, or should it be a data file? My hunch is it should be a single object, and that we should have a separate valid_data_files directory.
To support validation, we could store a random seed in data file's metadata. Then a validator can read the file while generating random objects from the seed and schema and check that the two match.
> We should probably do the same thing for valid/invalid protocols.
+1 We might also inlclude valid requests and responses.
> Create a shared schema test directory structure
> -----------------------------------------------
>
> Key: AVRO-230
> URL: https://issues.apache.org/jira/browse/AVRO-230
> Project: Avro
> Issue Type: New Feature
> Components: c, c++, java, python
> Reporter: Matt Massie
> Assignee: Matt Massie
> Fix For: 1.3.0
>
>
> This is an example of my proposed directory structure:
> * invalid_schemas/
> ** broken.json
> ** wrong.json
> * valid_data/
> ** foo_test/
> *** schema.json
> *** json_data/
> **** valid_json_test_data.json
> **** more_valid_json_test_data.json
> *** binary_data/
> **** valid_binary_test_data.bin
> **** more_test_data.bin
> ** bar_test/
> *** schema.json
> *** json_data/
> **** ...
> * invalid_data/
> ** baz_test/
> *** schema.json
> *** json_data/
> **** ...
> *** binary_data/
> **** ...
> This structure supports positive and negative tests for avro schemas, json data and binary data.
> * The "invalid_schema" directory holds a number of invalid schemas that should fail to parse.
> * The "valid_data" directory has a number of self-contained tests in separate directories. Each test directory is required to have a "schema.json" file that valid avro schema. The "json_data" and "binary_data" directories are optional for each test.
> * The "invalid_data" directory has the same rules as the "valid_data" directory. The data files should fail during tests (negative testing).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-230) Create a shared schema test directory
structure
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12851020#action_12851020 ]
Doug Cutting commented on AVRO-230:
-----------------------------------
> We can have things without random data involved and a static directory of known values that we can write tests to work against, no?
Yes, something would be much better than nothing here.
> Create a shared schema test directory structure
> -----------------------------------------------
>
> Key: AVRO-230
> URL: https://issues.apache.org/jira/browse/AVRO-230
> Project: Avro
> Issue Type: New Feature
> Components: c, c++, java, python
> Reporter: Matt Massie
> Assignee: Matt Massie
>
> This is an example of my proposed directory structure:
> * invalid_schemas/
> ** broken.json
> ** wrong.json
> * valid_data/
> ** foo_test/
> *** schema.json
> *** json_data/
> **** valid_json_test_data.json
> **** more_valid_json_test_data.json
> *** binary_data/
> **** valid_binary_test_data.bin
> **** more_test_data.bin
> ** bar_test/
> *** schema.json
> *** json_data/
> **** ...
> * invalid_data/
> ** baz_test/
> *** schema.json
> *** json_data/
> **** ...
> *** binary_data/
> **** ...
> This structure supports positive and negative tests for avro schemas, json data and binary data.
> * The "invalid_schema" directory holds a number of invalid schemas that should fail to parse.
> * The "valid_data" directory has a number of self-contained tests in separate directories. Each test directory is required to have a "schema.json" file that valid avro schema. The "json_data" and "binary_data" directories are optional for each test.
> * The "invalid_data" directory has the same rules as the "valid_data" directory. The data files should fail during tests (negative testing).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-230) Create a shared schema test directory
structure
Posted by "Jeff Hammerbacher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jeff Hammerbacher updated AVRO-230:
-----------------------------------
Fix Version/s: (was: 1.3.0)
> Create a shared schema test directory structure
> -----------------------------------------------
>
> Key: AVRO-230
> URL: https://issues.apache.org/jira/browse/AVRO-230
> Project: Avro
> Issue Type: New Feature
> Components: c, c++, java, python
> Reporter: Matt Massie
> Assignee: Matt Massie
>
> This is an example of my proposed directory structure:
> * invalid_schemas/
> ** broken.json
> ** wrong.json
> * valid_data/
> ** foo_test/
> *** schema.json
> *** json_data/
> **** valid_json_test_data.json
> **** more_valid_json_test_data.json
> *** binary_data/
> **** valid_binary_test_data.bin
> **** more_test_data.bin
> ** bar_test/
> *** schema.json
> *** json_data/
> **** ...
> * invalid_data/
> ** baz_test/
> *** schema.json
> *** json_data/
> **** ...
> *** binary_data/
> **** ...
> This structure supports positive and negative tests for avro schemas, json data and binary data.
> * The "invalid_schema" directory holds a number of invalid schemas that should fail to parse.
> * The "valid_data" directory has a number of self-contained tests in separate directories. Each test directory is required to have a "schema.json" file that valid avro schema. The "json_data" and "binary_data" directories are optional for each test.
> * The "invalid_data" directory has the same rules as the "valid_data" directory. The data files should fail during tests (negative testing).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-230) Create a shared schema test directory
structure
Posted by "Jeff Hammerbacher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12780480#action_12780480 ]
Jeff Hammerbacher commented on AVRO-230:
----------------------------------------
Love it! From working with the schemas you use for the C code: it would be nice to have some information about just what corner case the schema is meant to test. You've stuffed some information in the file name, and some in the JSON text itself, but it would be nice to have a more official way of encoding that information.
I'd vote for a "valid_schemas" directory too.
We should probably do the same thing for valid/invalid protocols.
> Create a shared schema test directory structure
> -----------------------------------------------
>
> Key: AVRO-230
> URL: https://issues.apache.org/jira/browse/AVRO-230
> Project: Avro
> Issue Type: New Feature
> Components: c, c++, java, python
> Reporter: Matt Massie
> Assignee: Matt Massie
> Fix For: 1.3.0
>
>
> This is an example of my proposed directory structure:
> * invalid_schemas/
> ** broken.json
> ** wrong.json
> * valid_data/
> ** foo_test/
> *** schema.json
> *** json_data/
> **** valid_json_test_data.json
> **** more_valid_json_test_data.json
> *** binary_data/
> **** valid_binary_test_data.bin
> **** more_test_data.bin
> ** bar_test/
> *** schema.json
> *** json_data/
> **** ...
> * invalid_data/
> ** baz_test/
> *** schema.json
> *** json_data/
> **** ...
> *** binary_data/
> **** ...
> This structure supports positive and negative tests for avro schemas, json data and binary data.
> * The "invalid_schema" directory holds a number of invalid schemas that should fail to parse.
> * The "valid_data" directory has a number of self-contained tests in separate directories. Each test directory is required to have a "schema.json" file that valid avro schema. The "json_data" and "binary_data" directories are optional for each test.
> * The "invalid_data" directory has the same rules as the "valid_data" directory. The data files should fail during tests (negative testing).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-230) Create a shared schema test directory
structure
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12780734#action_12780734 ]
Doug Cutting commented on AVRO-230:
-----------------------------------
> generate a random number and compare to the encoders stored value?
What I'm thinking of is that we provide somewhere a spec for a random data generator. This can be based on, e.g., a simple 32-bit Linear Congruential random number generator:
http://en.wikipedia.org/wiki/Linear_congruential_generator
This can be implemented in a few simple lines of code as the "raw" generator. Since small values should often be more common, we can define a biased generator by using the high bit (high bits are more random) of the next raw random number as a coin. Toss it until false, counting the tosses, then use that many high-order bits of the next raw random number as the biased value.
Then we specify for each kind of schema, how random data should be generated.
- int: take the next biased value. subtract it from 0 if the next raw value's high-bit is one.
- long: multiply the next two biased values, then compute sign as above.
- float/double: divide the next two biased values, compute sign as above.
- bytes: use next biased value as length, up to, e.g., 64k, then fill with high-byte of next raw value.
- string: use next biased value as length, up to, e.g., 64k, then fill with randomly selected [a-z].
- union: select branch next_raw % branch_count
- etc.
The remaining question is how to seed the LCG generator. My instinct is not to always start with a fixed seed, but rather should select a seed based on, e.g., the system time when a test file is created, and store that in each file's metadata.
> Create a shared schema test directory structure
> -----------------------------------------------
>
> Key: AVRO-230
> URL: https://issues.apache.org/jira/browse/AVRO-230
> Project: Avro
> Issue Type: New Feature
> Components: c, c++, java, python
> Reporter: Matt Massie
> Assignee: Matt Massie
> Fix For: 1.3.0
>
>
> This is an example of my proposed directory structure:
> * invalid_schemas/
> ** broken.json
> ** wrong.json
> * valid_data/
> ** foo_test/
> *** schema.json
> *** json_data/
> **** valid_json_test_data.json
> **** more_valid_json_test_data.json
> *** binary_data/
> **** valid_binary_test_data.bin
> **** more_test_data.bin
> ** bar_test/
> *** schema.json
> *** json_data/
> **** ...
> * invalid_data/
> ** baz_test/
> *** schema.json
> *** json_data/
> **** ...
> *** binary_data/
> **** ...
> This structure supports positive and negative tests for avro schemas, json data and binary data.
> * The "invalid_schema" directory holds a number of invalid schemas that should fail to parse.
> * The "valid_data" directory has a number of self-contained tests in separate directories. Each test directory is required to have a "schema.json" file that valid avro schema. The "json_data" and "binary_data" directories are optional for each test.
> * The "invalid_data" directory has the same rules as the "valid_data" directory. The data files should fail during tests (negative testing).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-230) Create a shared schema test directory
structure
Posted by "Matt Massie (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12780775#action_12780775 ]
Matt Massie commented on AVRO-230:
----------------------------------
Seeding with a fixed number is an awful idea now that I really think about it. Your approach is better.
> Create a shared schema test directory structure
> -----------------------------------------------
>
> Key: AVRO-230
> URL: https://issues.apache.org/jira/browse/AVRO-230
> Project: Avro
> Issue Type: New Feature
> Components: c, c++, java, python
> Reporter: Matt Massie
> Assignee: Matt Massie
> Fix For: 1.3.0
>
>
> This is an example of my proposed directory structure:
> * invalid_schemas/
> ** broken.json
> ** wrong.json
> * valid_data/
> ** foo_test/
> *** schema.json
> *** json_data/
> **** valid_json_test_data.json
> **** more_valid_json_test_data.json
> *** binary_data/
> **** valid_binary_test_data.bin
> **** more_test_data.bin
> ** bar_test/
> *** schema.json
> *** json_data/
> **** ...
> * invalid_data/
> ** baz_test/
> *** schema.json
> *** json_data/
> **** ...
> *** binary_data/
> **** ...
> This structure supports positive and negative tests for avro schemas, json data and binary data.
> * The "invalid_schema" directory holds a number of invalid schemas that should fail to parse.
> * The "valid_data" directory has a number of self-contained tests in separate directories. Each test directory is required to have a "schema.json" file that valid avro schema. The "json_data" and "binary_data" directories are optional for each test.
> * The "invalid_data" directory has the same rules as the "valid_data" directory. The data files should fail during tests (negative testing).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-230) Create a shared schema test directory
structure
Posted by "Bruce Mitchener (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12850458#action_12850458 ]
Bruce Mitchener commented on AVRO-230:
--------------------------------------
We need at least a simple version of this sooner rather than later (we aren't currently doing correct interop testing of the actual values of floats, doubles and such).
We can have things without random data involved and a static directory of known values that we can write tests to work against, no?
Keeping it simple would mean that someone could start knocking out some basic examples in an hour or two and then we'd just need a reader in each langauge to verify.
> Create a shared schema test directory structure
> -----------------------------------------------
>
> Key: AVRO-230
> URL: https://issues.apache.org/jira/browse/AVRO-230
> Project: Avro
> Issue Type: New Feature
> Components: c, c++, java, python
> Reporter: Matt Massie
> Assignee: Matt Massie
>
> This is an example of my proposed directory structure:
> * invalid_schemas/
> ** broken.json
> ** wrong.json
> * valid_data/
> ** foo_test/
> *** schema.json
> *** json_data/
> **** valid_json_test_data.json
> **** more_valid_json_test_data.json
> *** binary_data/
> **** valid_binary_test_data.bin
> **** more_test_data.bin
> ** bar_test/
> *** schema.json
> *** json_data/
> **** ...
> * invalid_data/
> ** baz_test/
> *** schema.json
> *** json_data/
> **** ...
> *** binary_data/
> **** ...
> This structure supports positive and negative tests for avro schemas, json data and binary data.
> * The "invalid_schema" directory holds a number of invalid schemas that should fail to parse.
> * The "valid_data" directory has a number of self-contained tests in separate directories. Each test directory is required to have a "schema.json" file that valid avro schema. The "json_data" and "binary_data" directories are optional for each test.
> * The "invalid_data" directory has the same rules as the "valid_data" directory. The data files should fail during tests (negative testing).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.