You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Gary Dusbabek (JIRA)" <ji...@apache.org> on 2011/01/18 15:03:43 UTC
[jira] Created: (CASSANDRA-2001) 0.6 migrations/schema
serializations are incompatible with trunk
0.6 migrations/schema serializations are incompatible with trunk
----------------------------------------------------------------
Key: CASSANDRA-2001
URL: https://issues.apache.org/jira/browse/CASSANDRA-2001
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Gary Dusbabek
Assignee: Gary Dusbabek
Priority: Minor
Fix For: 0.8
Two problems:
1. inserting replicate_on_write into the middle of the CfDef members created a problem with serialization.
2. merging the genavro files created a strange namespacing problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-2001) 0.7 migrations/schema
serializations are incompatible with trunk
Posted by "Stu Hood (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stu Hood updated CASSANDRA-2001:
--------------------------------
Attachment: 0004-Set-a-default-for-rep-on-write-and-revert-0001.txt
Attaching the patch discussed in IRC.
In summary: it is necessary to add a default with every new field, independent of whether you are using a union to make the field optional. Defaults are used in places where the writer's schema doesn't contain a field, but the reader's schema does. Unions are used to make fields optional: you can use a union together with a default to add a new optional field.
> 0.7 migrations/schema serializations are incompatible with trunk
> ----------------------------------------------------------------
>
> Key: CASSANDRA-2001
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2001
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: Gary Dusbabek
> Assignee: Gary Dusbabek
> Priority: Minor
> Fix For: 0.8
>
> Attachments: 0004-Set-a-default-for-rep-on-write-and-revert-0001.txt, data.zip, v1-0001-use-the-writer-schema-and-not-the-current-schema-when-.txt, v1-0002-get-rid-of-the-avro-db.migrations-classes-that-were-du.txt, v1-0003-fix-order-of-replicate_on_write-and-make-sure-it-s-not.txt
>
>
> Two problems:
> 1. inserting replicate_on_write into the middle of the CfDef members created a problem with serialization.
> 2. merging the genavro files created a strange namespacing problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-2001) 0.7 migrations/schema
serializations are incompatible with trunk
Posted by "Gary Dusbabek (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gary Dusbabek updated CASSANDRA-2001:
-------------------------------------
Summary: 0.7 migrations/schema serializations are incompatible with trunk (was: 0.6 migrations/schema serializations are incompatible with trunk)
> 0.7 migrations/schema serializations are incompatible with trunk
> ----------------------------------------------------------------
>
> Key: CASSANDRA-2001
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2001
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: Gary Dusbabek
> Assignee: Gary Dusbabek
> Priority: Minor
> Fix For: 0.8
>
>
> Two problems:
> 1. inserting replicate_on_write into the middle of the CfDef members created a problem with serialization.
> 2. merging the genavro files created a strange namespacing problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-2001) 0.7 migrations/schema
serializations are incompatible with trunk
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12983305#action_12983305 ]
Hudson commented on CASSANDRA-2001:
-----------------------------------
Integrated in Cassandra #676 (See [https://hudson.apache.org/hudson/job/Cassandra/676/])
fix order of replicate_on_write and make sure it's not null. patch by gdusbabek, reviewed by eevans. CASSANDRA-2001
get rid of the avro db.migrations classes that were duplicated elsewhere. patch by gdusbabek, reviewed by eevans. CASSANDRA-2001
use the writer schema and not the current schema when deserializing. patch by gdusbabek, reviewed by eevans. CASSANDRA-2001
> 0.7 migrations/schema serializations are incompatible with trunk
> ----------------------------------------------------------------
>
> Key: CASSANDRA-2001
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2001
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: Gary Dusbabek
> Assignee: Gary Dusbabek
> Priority: Minor
> Fix For: 0.8
>
> Attachments: v1-0001-use-the-writer-schema-and-not-the-current-schema-when-.txt, v1-0002-get-rid-of-the-avro-db.migrations-classes-that-were-du.txt, v1-0003-fix-order-of-replicate_on_write-and-make-sure-it-s-not.txt
>
>
> Two problems:
> 1. inserting replicate_on_write into the middle of the CfDef members created a problem with serialization.
> 2. merging the genavro files created a strange namespacing problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-2001) 0.7 migrations/schema
serializations are incompatible with trunk
Posted by "Gary Dusbabek (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gary Dusbabek updated CASSANDRA-2001:
-------------------------------------
Attachment: v1-0003-fix-order-of-replicate_on_write-and-make-sure-it-s-not.txt
v1-0002-get-rid-of-the-avro-db.migrations-classes-that-were-du.txt
v1-0001-use-the-writer-schema-and-not-the-current-schema-when-.txt
> 0.7 migrations/schema serializations are incompatible with trunk
> ----------------------------------------------------------------
>
> Key: CASSANDRA-2001
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2001
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: Gary Dusbabek
> Assignee: Gary Dusbabek
> Priority: Minor
> Fix For: 0.8
>
> Attachments: v1-0001-use-the-writer-schema-and-not-the-current-schema-when-.txt, v1-0002-get-rid-of-the-avro-db.migrations-classes-that-were-du.txt, v1-0003-fix-order-of-replicate_on_write-and-make-sure-it-s-not.txt
>
>
> Two problems:
> 1. inserting replicate_on_write into the middle of the CfDef members created a problem with serialization.
> 2. merging the genavro files created a strange namespacing problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-2001) 0.7 migrations/schema
serializations are incompatible with trunk
Posted by "Gary Dusbabek (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gary Dusbabek updated CASSANDRA-2001:
-------------------------------------
Attachment: data.zip
schema from a vanilla 0.7 node.
> 0.7 migrations/schema serializations are incompatible with trunk
> ----------------------------------------------------------------
>
> Key: CASSANDRA-2001
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2001
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: Gary Dusbabek
> Assignee: Gary Dusbabek
> Priority: Minor
> Fix For: 0.8
>
> Attachments: data.zip, v1-0001-use-the-writer-schema-and-not-the-current-schema-when-.txt, v1-0002-get-rid-of-the-avro-db.migrations-classes-that-were-du.txt, v1-0003-fix-order-of-replicate_on_write-and-make-sure-it-s-not.txt
>
>
> Two problems:
> 1. inserting replicate_on_write into the middle of the CfDef members created a problem with serialization.
> 2. merging the genavro files created a strange namespacing problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-2001) 0.7 migrations/schema
serializations are incompatible with trunk
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12983431#action_12983431 ]
Hudson commented on CASSANDRA-2001:
-----------------------------------
Integrated in Cassandra #677 (See [https://hudson.apache.org/hudson/job/Cassandra/677/])
set a default for replicate_on_write. patch by stuhood and gdusbabek. CASSANDRA-2001
back out of 2001. patch by gdusbabek. CASSANDRA-2001
> 0.7 migrations/schema serializations are incompatible with trunk
> ----------------------------------------------------------------
>
> Key: CASSANDRA-2001
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2001
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: Gary Dusbabek
> Assignee: Gary Dusbabek
> Priority: Minor
> Fix For: 0.8
>
> Attachments: 0004-Set-a-default-for-rep-on-write-and-revert-0001.txt, data.zip, v1-0001-use-the-writer-schema-and-not-the-current-schema-when-.txt, v1-0002-get-rid-of-the-avro-db.migrations-classes-that-were-du.txt, v1-0003-fix-order-of-replicate_on_write-and-make-sure-it-s-not.txt
>
>
> Two problems:
> 1. inserting replicate_on_write into the middle of the CfDef members created a problem with serialization.
> 2. merging the genavro files created a strange namespacing problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-2001) 0.7 migrations/schema
serializations are incompatible with trunk
Posted by "Eric Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12983288#action_12983288 ]
Eric Evans commented on CASSANDRA-2001:
---------------------------------------
Just for posterity sake: CASSANDRA-926 moved the remaining Avro records from o.a.c.avro elsewhere, to packages that made it obvious which components were still using them (but obviously, created a bug in the process).
+1 on this patchset though (with or without the o.a.c.db.migration.avro -> o.a.c.db.avro move).
> 0.7 migrations/schema serializations are incompatible with trunk
> ----------------------------------------------------------------
>
> Key: CASSANDRA-2001
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2001
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: Gary Dusbabek
> Assignee: Gary Dusbabek
> Priority: Minor
> Fix For: 0.8
>
> Attachments: v1-0001-use-the-writer-schema-and-not-the-current-schema-when-.txt, v1-0002-get-rid-of-the-avro-db.migrations-classes-that-were-du.txt, v1-0003-fix-order-of-replicate_on_write-and-make-sure-it-s-not.txt
>
>
> Two problems:
> 1. inserting replicate_on_write into the middle of the CfDef members created a problem with serialization.
> 2. merging the genavro files created a strange namespacing problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-2001) 0.7 migrations/schema
serializations are incompatible with trunk
Posted by "Gary Dusbabek (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12983385#action_12983385 ]
Gary Dusbabek commented on CASSANDRA-2001:
------------------------------------------
backed out of the original fix and committed the one-liner that sets a default on CfDef.replicate_on_write.
> 0.7 migrations/schema serializations are incompatible with trunk
> ----------------------------------------------------------------
>
> Key: CASSANDRA-2001
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2001
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: Gary Dusbabek
> Assignee: Gary Dusbabek
> Priority: Minor
> Fix For: 0.8
>
> Attachments: 0004-Set-a-default-for-rep-on-write-and-revert-0001.txt, data.zip, v1-0001-use-the-writer-schema-and-not-the-current-schema-when-.txt, v1-0002-get-rid-of-the-avro-db.migrations-classes-that-were-du.txt, v1-0003-fix-order-of-replicate_on_write-and-make-sure-it-s-not.txt
>
>
> Two problems:
> 1. inserting replicate_on_write into the middle of the CfDef members created a problem with serialization.
> 2. merging the genavro files created a strange namespacing problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (CASSANDRA-2001) 0.7 migrations/schema
serializations are incompatible with trunk
Posted by "Gary Dusbabek (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12983274#action_12983274 ]
Gary Dusbabek edited comment on CASSANDRA-2001 at 1/18/11 12:20 PM:
--------------------------------------------------------------------
also fixes the incorrect schema that was used when deserializing an object.
was (Author: gdusbabek):
fixes the incorrect schema that was used when deserializing an object.
> 0.7 migrations/schema serializations are incompatible with trunk
> ----------------------------------------------------------------
>
> Key: CASSANDRA-2001
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2001
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: Gary Dusbabek
> Assignee: Gary Dusbabek
> Priority: Minor
> Fix For: 0.8
>
> Attachments: v1-0001-use-the-writer-schema-and-not-the-current-schema-when-.txt, v1-0002-get-rid-of-the-avro-db.migrations-classes-that-were-du.txt, v1-0003-fix-order-of-replicate_on_write-and-make-sure-it-s-not.txt
>
>
> Two problems:
> 1. inserting replicate_on_write into the middle of the CfDef members created a problem with serialization.
> 2. merging the genavro files created a strange namespacing problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-2001) 0.7 migrations/schema
serializations are incompatible with trunk
Posted by "Gary Dusbabek (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12983364#action_12983364 ]
Gary Dusbabek commented on CASSANDRA-2001:
------------------------------------------
If you bypass the ClassCastException that 926 introduced, you end up with an avro error:
org.apache.avro.AvroTypeException: Found {"type":"record","name":"CfDef","namespace":"org.apache.cassandra.avro","fields":[{"name":"keyspace","type":"string"},{"name":"name","type":"string"},{"name":"column_type","type":["string","null"]},{"name":"comparator_type","type":["string","null"]},{"name":"subcomparator_type","type":["string","null"]},{"name":"comment","type":["string","null"]},{"name":"row_cache_size","type":["double","null"]},{"name":"key_cache_size","type":["double","null"]},{"name":"read_repair_chance","type":["double","null"]},{"name":"gc_grace_seconds","type":["int","null"]},{"name":"default_validation_class","type":["null","string"],"default":null},{"name":"min_compaction_threshold","type":["null","int"],"default":null},{"name":"max_compaction_threshold","type":["null","int"],"default":null},{"name":"row_cache_save_period_in_seconds","type":["int","null"],"default":0},{"name":"key_cache_save_period_in_seconds","type":["int","null"],"default":3600},{"name":"memtable_flush_after_mins","type":["int","null"],"default":60},{"name":"memtable_throughput_in_mb","type":["null","int"],"default":null},{"name":"memtable_operations_in_millions","type":["null","double"],"default":null},{"name":"id","type":["int","null"]},{"name":"column_metadata","type":[{"type":"array","items":{"type":"record","name":"ColumnDef","fields":[{"name":"name","type":"bytes"},{"name":"validation_class","type":"string"},{"name":"index_type","type":[{"type":"enum","name":"IndexType","symbols":["KEYS"],"aliases":["org.apache.cassandra.config.avro.IndexType"]},"null"]},{"name":"index_name","type":["string","null"]}]}},"null"]}]}, expecting {"type":"record","name":"CfDef","namespace":"org.apache.cassandra.avro","fields":[{"name":"keyspace","type":"string"},{"name":"name","type":"string"},{"name":"column_type","type":["string","null"]},{"name":"comparator_type","type":["string","null"]},{"name":"subcomparator_type","type":["string","null"]},{"name":"comment","type":["string","null"]},{"name":"row_cache_size","type":["double","null"]},{"name":"key_cache_size","type":["double","null"]},{"name":"read_repair_chance","type":["double","null"]},{"name":"gc_grace_seconds","type":["int","null"]},{"name":"default_validation_class","type":["null","string"],"default":null},{"name":"min_compaction_threshold","type":["null","int"],"default":null},{"name":"max_compaction_threshold","type":["null","int"],"default":null},{"name":"row_cache_save_period_in_seconds","type":["int","null"],"default":0},{"name":"key_cache_save_period_in_seconds","type":["int","null"],"default":3600},{"name":"memtable_flush_after_mins","type":["int","null"],"default":60},{"name":"memtable_throughput_in_mb","type":["null","int"],"default":null},{"name":"memtable_operations_in_millions","type":["null","double"],"default":null},{"name":"id","type":["int","null"]},{"name":"column_metadata","type":[{"type":"array","items":{"type":"record","name":"ColumnDef","fields":[{"name":"name","type":"bytes"},{"name":"validation_class","type":"string"},{"name":"index_type","type":[{"type":"enum","name":"IndexType","symbols":["KEYS"],"aliases":["org.apache.cassandra.config.avro.IndexType"]},"null"]},{"name":"index_name","type":["string","null"]}],"aliases":["org.apache.cassandra.config.avro.ColumnDef"]}},"null"]},{"name":"replicate_on_write","type":["boolean","null"]}],"aliases":["org.apache.cassandra.config.avro.CfDef"]}
basically, avro was expecting to see replicate_on_write.
> 0.7 migrations/schema serializations are incompatible with trunk
> ----------------------------------------------------------------
>
> Key: CASSANDRA-2001
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2001
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: Gary Dusbabek
> Assignee: Gary Dusbabek
> Priority: Minor
> Fix For: 0.8
>
> Attachments: data.zip, v1-0001-use-the-writer-schema-and-not-the-current-schema-when-.txt, v1-0002-get-rid-of-the-avro-db.migrations-classes-that-were-du.txt, v1-0003-fix-order-of-replicate_on_write-and-make-sure-it-s-not.txt
>
>
> Two problems:
> 1. inserting replicate_on_write into the middle of the CfDef members created a problem with serialization.
> 2. merging the genavro files created a strange namespacing problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-2001) 0.6 migrations/schema
serializations are incompatible with trunk
Posted by "Gary Dusbabek (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12983196#action_12983196 ]
Gary Dusbabek commented on CASSANDRA-2001:
------------------------------------------
fwiw CASSANDRA-1923 contains unit tests that will catch these kinds of problems in the future.
> 0.6 migrations/schema serializations are incompatible with trunk
> ----------------------------------------------------------------
>
> Key: CASSANDRA-2001
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2001
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: Gary Dusbabek
> Assignee: Gary Dusbabek
> Priority: Minor
> Fix For: 0.8
>
>
> Two problems:
> 1. inserting replicate_on_write into the middle of the CfDef members created a problem with serialization.
> 2. merging the genavro files created a strange namespacing problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.