You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Sylvain Lebresne (JIRA)" <ji...@apache.org> on 2011/01/05 16:48:46 UTC
[jira] Created: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Use UUID as node identifiers in counters instead of IP addresses
-----------------------------------------------------------------
Key: CASSANDRA-1938
URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Fix For: 0.8
The use of IP addresses as node identifiers in the partition of a given
counter is fragile. Changes of the node's IP addresses can result in data
loss. This patch proposes to use UUIDs instead.
NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sylvain Lebresne updated CASSANDRA-1938:
----------------------------------------
Attachment: 0003-Thrift-change-to-CfDef.patch
0002-Merge-old-shard-locally.patch
0001-Use-uuid-instead-of-IP-for-counters.patch
Rebased patch attached. I've also run more test, in particular some boostrap followed by decomission with and without cleanup in between (and writing/reading counter during all this) and those tests pass (they don't with current trunk). So confidence++ on this.
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 0001-Use-uuid-instead-of-IP-for-counters.patch, 0002-Merge-old-shard-locally.patch, 0003-Thrift-change-to-CfDef.patch, 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sylvain Lebresne updated CASSANDRA-1938:
----------------------------------------
Attachment: 0002-Merge-old-shard-locally.patch
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 0001-Use-uuid-instead-of-IP-for-counters.patch, 0002-Merge-old-shard-locally.patch, 0003-Thrift-change-to-CfDef.patch, 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13003451#comment-13003451 ]
Sylvain Lebresne commented on CASSANDRA-1938:
---------------------------------------------
The problem is with the updateDigest part of PreCompactedRow. For this patch, we need it to use column updateDigest() function because for counterColumns it's not trivial.
For PreCompactedRow, this mean we should deserialize fully at least for counter CF && for updateDigest. I felt the best way to do this was to make PreCompactedRow store the deserialized and resolved resulting CF and have write and updateDigest do their thing, instead of storing directly the serialized result.
We could add more special casing for counters, but I actually think that having updateDigest not use the column updateDigest functions is a mistake in the first place. It does kill the echoRow optimization, but I'm not sure how useful it is in the first place (especially now that the 'clear tombstone on minor compaction' optimization make the echoRow one trigger even less often).
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 0001-Use-uuid-instead-of-IP-for-counters.patch, 0002-Merge-old-shard-locally.patch, 0003-Thrift-change-to-CfDef.patch, 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sylvain Lebresne updated CASSANDRA-1938:
----------------------------------------
Attachment: (was: 0003-Thrift-change-to-CfDef.patch)
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sylvain Lebresne updated CASSANDRA-1938:
----------------------------------------
Attachment: 0002-Merge-old-shard-locally.patch
0001-Use-uuid-instead-of-IP-for-counters.patch
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 0001-Use-uuid-instead-of-IP-for-counters.patch, 0002-Merge-old-shard-locally.patch, 0003-Thrift-change-to-CfDef.patch, 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sylvain Lebresne updated CASSANDRA-1938:
----------------------------------------
Attachment: (was: 0002-Merge-old-shard-locally.patch)
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 0001-Use-uuid-instead-of-IP-for-counters.patch, 0003-Thrift-change-to-CfDef.patch, 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "T Jake Luciani (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13005310#comment-13005310 ]
T Jake Luciani commented on CASSANDRA-1938:
-------------------------------------------
Ok cluster testing complete and all looks good!
I just noticed a SerializationTest started failing though, Can you take a look
{code}
[junit] Testcase: testRowMutationRead(org.apache.cassandra.db.SerializationsTest): FAILED
[junit] null
[junit] junit.framework.AssertionFailedError
[junit] at org.apache.cassandra.io.sstable.SSTable.estimateRowsFromIndex(SSTable.java:233)
[junit] at org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:286)
[junit] at org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:191)
[junit] at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:224)
[junit] at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:506)
[junit] at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:487)
[junit] at org.apache.cassandra.db.Table.initCf(Table.java:358)
[junit] at org.apache.cassandra.db.Table.<init>(Table.java:295)
[junit] at org.apache.cassandra.db.Table.open(Table.java:112)
[junit] at org.apache.cassandra.db.ColumnFamilySerializer.deserializeColumns(ColumnFamilySerializer.java:132)
[junit] at org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:125)
[junit] at org.apache.cassandra.db.RowMutation$RowMutationSerializer.deserialize(RowMutation.java:365)
[junit] at org.apache.cassandra.db.SerializationsTest.testRowMutationRead(SerializationsTest.java:219)
{code}
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 0001-Use-uuid-instead-of-IP-for-counters.patch, 0002-Merge-old-shard-locally.patch, 0003-Thrift-change-to-CfDef.patch, 1938-rebase-and-fix-v1.txt, 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "T Jake Luciani (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13004574#comment-13004574 ]
T Jake Luciani commented on CASSANDRA-1938:
-------------------------------------------
I'm interested in seeing more of the performance impact of this change, I created CASSANDRA-2299 to help.
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 0001-Use-uuid-instead-of-IP-for-counters.patch, 0002-Merge-old-shard-locally.patch, 0003-Thrift-change-to-CfDef.patch, 1938-rebase-and-fix-v1.txt, 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "Ryan King (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13003450#comment-13003450 ]
Ryan King commented on CASSANDRA-1938:
--------------------------------------
it was removed because it doesn't work across changes to the file formats.
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 0001-Use-uuid-instead-of-IP-for-counters.patch, 0002-Merge-old-shard-locally.patch, 0003-Thrift-change-to-CfDef.patch, 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13003995#comment-13003995 ]
Jonathan Ellis commented on CASSANDRA-1938:
-------------------------------------------
bq. I'm not super convinced that the echoRow 'optimisation' is really that useful anymore... during compaction, we echo a row if it's only in one of the sstable we're compacting but it also exists in a sstable we are not compacting
I'll buy that. +1 for removing echoing outside cleanup.
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 0001-Use-uuid-instead-of-IP-for-counters.patch, 0002-Merge-old-shard-locally.patch, 0003-Thrift-change-to-CfDef.patch, 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sylvain Lebresne updated CASSANDRA-1938:
----------------------------------------
Attachment: (was: 0001-Use-uuid-instead-of-IP-for-counters.patch)
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 0001-Use-uuid-instead-of-IP-for-counters.patch, 0002-Merge-old-shard-locally.patch, 0003-Thrift-change-to-CfDef.patch, 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "T Jake Luciani (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
T Jake Luciani updated CASSANDRA-1938:
--------------------------------------
Attachment: 1938-rebase-and-fix-v1.txt
Rebased with trunk...
During my testing I discovered a NPE when converting from thrift -> avro migration that's fixed. I need to do more testing but looks good so far.
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 0001-Use-uuid-instead-of-IP-for-counters.patch, 0002-Merge-old-shard-locally.patch, 0003-Thrift-change-to-CfDef.patch, 1938-rebase-and-fix-v1.txt, 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "T Jake Luciani (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13003458#comment-13003458 ]
T Jake Luciani commented on CASSANDRA-1938:
-------------------------------------------
Ryan we left it in incase the compaction was running on a "latest" version sstable.
Is there anyway to deal with this by tracking the CounterColumns in -Statistics.db?
I would imagine most people would not be using counters so seems like a waste to throw away echoRow for the non-counter case.
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 0001-Use-uuid-instead-of-IP-for-counters.patch, 0002-Merge-old-shard-locally.patch, 0003-Thrift-change-to-CfDef.patch, 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "T Jake Luciani (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002690#comment-13002690 ]
T Jake Luciani commented on CASSANDRA-1938:
-------------------------------------------
I get the following error when compiling:
[javac] /Users/jake/workspace/cassandra-git/src/java/org/apache/cassandra/service/StorageService.java:1243: package NodeId does not exist
[javac] NodeId.OneShotRenewer nodeIdRenewer = new NodeId.OneShotRenewer();
[javac] ^
[javac] /Users/jake/workspace/cassandra-git/src/java/org/apache/cassandra/service/StorageService.java:1243: package NodeId does not exist
[javac] NodeId.OneShotRenewer nodeIdRenewer = new NodeId.OneShotRenewer();
[javac] ^
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 0001-Use-uuid-instead-of-IP-for-counters.patch, 0002-Merge-old-shard-locally.patch, 0003-Thrift-change-to-CfDef.patch, 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "T Jake Luciani (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13005319#comment-13005319 ]
T Jake Luciani commented on CASSANDRA-1938:
-------------------------------------------
+1, nice work
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 0001-Use-uuid-instead-of-IP-for-counters.patch, 0002-Merge-old-shard-locally.patch, 0003-Thrift-change-to-CfDef.patch, 1938-rebase-and-fix-v1.txt, 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sylvain Lebresne updated CASSANDRA-1938:
----------------------------------------
Attachment: (was: 0001-Use-uuid-instead-of-IP-for-counters.patch)
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "T Jake Luciani (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002631#comment-13002631 ]
T Jake Luciani commented on CASSANDRA-1938:
-------------------------------------------
PNYARB - Patch Needs Yet Another Rebase
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 0001-Use-uuid-instead-of-IP-for-counters.patch, 0002-Merge-old-shard-locally.patch, 0003-Thrift-change-to-CfDef.patch, 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sylvain Lebresne updated CASSANDRA-1938:
----------------------------------------
Attachment: (was: 0001-Use-uuid-instead-of-IP-for-counters.patch)
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 1938-rebase-and-fix-v1.txt, 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sylvain Lebresne updated CASSANDRA-1938:
----------------------------------------
Attachment: (was: 0002-Merge-old-shard-locally.patch)
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 1938-rebase-and-fix-v1.txt, 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sylvain Lebresne updated CASSANDRA-1938:
----------------------------------------
Attachment: (was: 0003-Thrift-change-to-CfDef.patch)
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 0001-Use-uuid-instead-of-IP-for-counters.patch, 0002-Merge-old-shard-locally.patch, 0003-Thrift-change-to-CfDef.patch, 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "T Jake Luciani (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
T Jake Luciani updated CASSANDRA-1938:
--------------------------------------
Comment: was deleted
(was: I get the following error when compiling:
[javac] /Users/jake/workspace/cassandra-git/src/java/org/apache/cassandra/service/StorageService.java:1243: package NodeId does not exist
[javac] NodeId.OneShotRenewer nodeIdRenewer = new NodeId.OneShotRenewer();
[javac] ^
[javac] /Users/jake/workspace/cassandra-git/src/java/org/apache/cassandra/service/StorageService.java:1243: package NodeId does not exist
[javac] NodeId.OneShotRenewer nodeIdRenewer = new NodeId.OneShotRenewer();
[javac] ^
)
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 0001-Use-uuid-instead-of-IP-for-counters.patch, 0002-Merge-old-shard-locally.patch, 0003-Thrift-change-to-CfDef.patch, 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sylvain Lebresne updated CASSANDRA-1938:
----------------------------------------
Attachment: (was: 0002-Merge-old-shard-locally.patch)
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 0001-Use-uuid-instead-of-IP-for-counters.patch, 0002-Merge-old-shard-locally.patch, 0003-Thrift-change-to-CfDef.patch, 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sylvain Lebresne updated CASSANDRA-1938:
----------------------------------------
Attachment: 1938_discussion
0001-Use-uuid-instead-of-IP-for-counters.patch
Patch attached. It passes basic testing (unit tests, system test and 'throwing a bunch of increments to a 3 nodes cluster' test).
I don't think this is a trivial patch (though I could be stupid) so the more people have a look, the merrier and all feedback is welcome. I've also attached a companion text file with some (lengthy but hopefully helpful) explanations of how it works and which problems it solves (and how).
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 0001-Use-uuid-instead-of-IP-for-counters.patch, 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sylvain Lebresne updated CASSANDRA-1938:
----------------------------------------
Attachment: (was: 0002-Merge-old-shard-locally.patch)
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 0001-Use-uuid-instead-of-IP-for-counters.patch, 0002-Merge-old-shard-locally.patch, 0003-Thrift-change-to-CfDef.patch, 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13006647#comment-13006647 ]
Hudson commented on CASSANDRA-1938:
-----------------------------------
Integrated in Cassandra #776 (See [https://hudson.apache.org/hudson/job/Cassandra/776/])
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 0001-Use-uuid-instead-of-IP-for-counters.patch, 0002-Merge-old-shard-locally.patch, 0003-Thrift-change-to-CfDef.patch, 1938-rebase-and-fix-v1.txt, 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13003863#comment-13003863 ]
Sylvain Lebresne commented on CASSANDRA-1938:
---------------------------------------------
bq. Is there anyway to deal with this by tracking the CounterColumns in -Statistics.db?
We don't even have to go there. We can have special casing for counter CFs if need be. But it won't be super pretty. In particular, you'll have to special case both PreCompactedRow and LazyCompactedRow. Because the main point is to use the columns updateDigest() function to compute the preCompactedRow digests instead of using the raw bytes. If you special case here, you'll have to mirror this in the lazy case, that is, use the columns updateDigest in the counter case, but the raw bytes otherwise.
However, what I'm trying to say is that I'm not super convinced that the echoRow 'optimisation' is really that useful anymore (I'm not talking about cleanup where echoing is useful and not changed by this patch). But during compaction, we echo a row if it's only in one of the sstable we're compacting *but* it also exists in a sstable we are not compacting (otherwise we still deserialize for tombstone reclaiming). I would imagine we have either rows that are often updated (in which case it will be rare to have only 1 of the sstable we compact containing it) or rows that are barely updated (in which case we'll still deserialize for tombstone reclaiming most of the time).
I'm also not sure we'll keep this 'optimization' forever anyway. If we add checksums for example (which we should imho, sooner than later), echoing data may not be desirable.
So given all this and given that even for the (I believe) rare cases where it is useful, it is not in a critical path, I'd advise against polluting the code for this.
That being said, if I'm the only one to feel that way, it's doable.
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 0001-Use-uuid-instead-of-IP-for-counters.patch, 0002-Merge-old-shard-locally.patch, 0003-Thrift-change-to-CfDef.patch, 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sylvain Lebresne updated CASSANDRA-1938:
----------------------------------------
Attachment: 0003-Thrift-change-to-CfDef.patch
0002-Merge-old-shard-locally.patch
0001-Use-uuid-instead-of-IP-for-counters.patch
Rebased to current trunk (svn rev 1080098)
I'll also try to keep https://github.com/pcmanus/cassandra/tree/1938_counters_uuid updated (aka, rebased to trunk).
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 0001-Use-uuid-instead-of-IP-for-counters.patch, 0002-Merge-old-shard-locally.patch, 0003-Thrift-change-to-CfDef.patch, 1938-rebase-and-fix-v1.txt, 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13005315#comment-13005315 ]
Jonathan Ellis commented on CASSANDRA-1938:
-------------------------------------------
that's been failing for a while, it's not this patch causing it.
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 0001-Use-uuid-instead-of-IP-for-counters.patch, 0002-Merge-old-shard-locally.patch, 0003-Thrift-change-to-CfDef.patch, 1938-rebase-and-fix-v1.txt, 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12977800#action_12977800 ]
Jonathan Ellis commented on CASSANDRA-1938:
-------------------------------------------
Will this also break the on-disk ring persistence we added for CASSANDRA-1518?
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sylvain Lebresne updated CASSANDRA-1938:
----------------------------------------
Attachment: 0003-Thrift-change-to-CfDef.patch
0002-Merge-old-shard-locally.patch
0001-Use-uuid-instead-of-IP-for-counters.patch
Rebased
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 0001-Use-uuid-instead-of-IP-for-counters.patch, 0002-Merge-old-shard-locally.patch, 0003-Thrift-change-to-CfDef.patch, 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sylvain Lebresne updated CASSANDRA-1938:
----------------------------------------
Attachment: (was: 0002-Merge-old-shard-locally.patch)
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "T Jake Luciani (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13003443#comment-13003443 ]
T Jake Luciani commented on CASSANDRA-1938:
-------------------------------------------
Trying to merge this with latest trunk and I noticed you removed the echoRow optimization from PrecompactedRow is this no longer possible?
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 0001-Use-uuid-instead-of-IP-for-counters.patch, 0002-Merge-old-shard-locally.patch, 0003-Thrift-change-to-CfDef.patch, 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sylvain Lebresne updated CASSANDRA-1938:
----------------------------------------
Attachment: (was: 0001-Use-uuid-instead-of-IP-for-counters.patch)
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sylvain Lebresne updated CASSANDRA-1938:
----------------------------------------
Attachment: 0003-Thrift-change-to-CfDef.patch
0002-Merge-old-shard-locally.patch
0001-Use-uuid-instead-of-IP-for-counters.patch
Re-attaching rebased patch.
The second patch implements a solution to merge old nodeId that has been renewed, fixing for the most part the main drawback of the previous patch (counter context should be kept reasonably small). It's honestly a bit under-tested, I've tested write and read, but I haven't tested more complicated scenario involving cluster membership changes. I'll get at it but attaching in the hope to get feedback sooner than later.
Last patch is just modifications to the thrift generated files implied by the second patch.
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 0001-Use-uuid-instead-of-IP-for-counters.patch, 0002-Merge-old-shard-locally.patch, 0003-Thrift-change-to-CfDef.patch, 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sylvain Lebresne updated CASSANDRA-1938:
----------------------------------------
Attachment: (was: 0003-Thrift-change-to-CfDef.patch)
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 1938-rebase-and-fix-v1.txt, 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12977806#action_12977806 ]
Sylvain Lebresne commented on CASSANDRA-1938:
---------------------------------------------
I'd have to look at CASSANDRA-1518 with more details, but for this ticket, I intend to keep those node identifiers strictly local, they will not get gossiped. So I expect that no, it won't beak on-disk ring persistence. I think we may have to gossip them at some point however to deal with ever increasing contexts, but I'm not yet completely clear on that and I don't think it's an urgent matter.
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1938) Use UUID as node identifiers in
counters instead of IP addresses
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sylvain Lebresne updated CASSANDRA-1938:
----------------------------------------
Attachment: (was: 0001-Use-uuid-instead-of-IP-for-counters.patch)
> Use UUID as node identifiers in counters instead of IP addresses
> -----------------------------------------------------------------
>
> Key: CASSANDRA-1938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1938
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments: 0001-Use-uuid-instead-of-IP-for-counters.patch, 0002-Merge-old-shard-locally.patch, 0003-Thrift-change-to-CfDef.patch, 1938_discussion
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> The use of IP addresses as node identifiers in the partition of a given
> counter is fragile. Changes of the node's IP addresses can result in data
> loss. This patch proposes to use UUIDs instead.
> NOTE: this breaks the on-disk file format (for counters)
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira