You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Jonathan Ellis (JIRA)" <ji...@apache.org> on 2011/07/19 18:52:58 UTC
[jira] [Created] (CASSANDRA-2920) Remove SSTableWriter.Builder
Remove SSTableWriter.Builder
----------------------------
Key: CASSANDRA-2920
URL: https://issues.apache.org/jira/browse/CASSANDRA-2920
Project: Cassandra
Issue Type: Task
Components: Core
Affects Versions: 1.0
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
Fix For: 1.0
CASSANDRA-2677 left Builder-oriented streaming in for backwards compatibility, but since we're going to change the sstable format for 1.0, and Builder doesn't work across format changes, there's no point in leaving Builder around.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2920) Remove SSTableWriter.Builder
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13067996#comment-13067996 ]
Jonathan Ellis commented on CASSANDRA-2920:
-------------------------------------------
bq. I can work on this in the next few days?
That would be great.
> Remove SSTableWriter.Builder
> ----------------------------
>
> Key: CASSANDRA-2920
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2920
> Project: Cassandra
> Issue Type: Task
> Components: Core
> Affects Versions: 1.0
> Reporter: Jonathan Ellis
> Assignee: Stu Hood
> Priority: Minor
> Fix For: 1.0
>
> Attachments: 2920.txt
>
>
> CASSANDRA-2677 left Builder-oriented streaming in for backwards compatibility, but since we're going to change the sstable format for 1.0, and Builder doesn't work across format changes, there's no point in leaving Builder around.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2920) Remove SSTableWriter.Builder
Posted by "Stu Hood (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stu Hood updated CASSANDRA-2920:
--------------------------------
Attachment: 2920-v3-0002-Fix-reference-counting-add-sstable-comparison-and-port.txt
2920-v3-0001.txt
Rebased to 2920-v3, and added patch 0002 to port Builder related tests to streaming tests.
0002 exposed a bug in Counter timestamp handling: see the changes to SSTableWriter and IncomingStreamReader.
> Remove SSTableWriter.Builder
> ----------------------------
>
> Key: CASSANDRA-2920
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2920
> Project: Cassandra
> Issue Type: Task
> Components: Core
> Affects Versions: 1.0
> Reporter: Jonathan Ellis
> Assignee: Stu Hood
> Priority: Minor
> Fix For: 1.0
>
> Attachments: 2920-v2.txt, 2920-v3-0001.txt, 2920-v3-0002-Fix-reference-counting-add-sstable-comparison-and-port.txt, 2920.txt
>
>
> CASSANDRA-2677 left Builder-oriented streaming in for backwards compatibility, but since we're going to change the sstable format for 1.0, and Builder doesn't work across format changes, there's no point in leaving Builder around.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2920) Remove SSTableWriter.Builder
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13070938#comment-13070938 ]
Jonathan Ellis commented on CASSANDRA-2920:
-------------------------------------------
committed, thanks!
> Remove SSTableWriter.Builder
> ----------------------------
>
> Key: CASSANDRA-2920
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2920
> Project: Cassandra
> Issue Type: Task
> Components: Core
> Affects Versions: 1.0
> Reporter: Jonathan Ellis
> Assignee: Stu Hood
> Priority: Minor
> Fix For: 1.0
>
> Attachments: 2920-v2.txt, 2920-v3-0001.txt, 2920-v3-0002-Fix-reference-counting-add-sstable-comparison-and-port.txt, 2920.txt
>
>
> CASSANDRA-2677 left Builder-oriented streaming in for backwards compatibility, but since we're going to change the sstable format for 1.0, and Builder doesn't work across format changes, there's no point in leaving Builder around.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2920) Remove SSTableWriter.Builder
Posted by "Stu Hood (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13067987#comment-13067987 ]
Stu Hood commented on CASSANDRA-2920:
-------------------------------------
Easiest way is probably to subclass StreamingTransferTest for different column families and column implementations.
> Remove SSTableWriter.Builder
> ----------------------------
>
> Key: CASSANDRA-2920
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2920
> Project: Cassandra
> Issue Type: Task
> Components: Core
> Affects Versions: 1.0
> Reporter: Jonathan Ellis
> Assignee: Stu Hood
> Priority: Minor
> Fix For: 1.0
>
> Attachments: 2920.txt
>
>
> CASSANDRA-2677 left Builder-oriented streaming in for backwards compatibility, but since we're going to change the sstable format for 1.0, and Builder doesn't work across format changes, there's no point in leaving Builder around.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2920) Remove SSTableWriter.Builder
Posted by "Yuki Morishita (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13069387#comment-13069387 ]
Yuki Morishita commented on CASSANDRA-2920:
-------------------------------------------
In previous comment, I wrote that estimatedKeys should be greater than 0. This is true if source node version is >= 1.0. Isn't there a situation that older version node streams to new one i.e. during rolling upgrade? In that situation, estimatedKeys is always 0.
> Remove SSTableWriter.Builder
> ----------------------------
>
> Key: CASSANDRA-2920
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2920
> Project: Cassandra
> Issue Type: Task
> Components: Core
> Affects Versions: 1.0
> Reporter: Jonathan Ellis
> Assignee: Stu Hood
> Priority: Minor
> Fix For: 1.0
>
> Attachments: 2920-v2.txt, 2920.txt
>
>
> CASSANDRA-2677 left Builder-oriented streaming in for backwards compatibility, but since we're going to change the sstable format for 1.0, and Builder doesn't work across format changes, there's no point in leaving Builder around.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2920) Remove SSTableWriter.Builder
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-2920:
--------------------------------------
Attachment: 2920-v2.txt
good points, v2 attached
> Remove SSTableWriter.Builder
> ----------------------------
>
> Key: CASSANDRA-2920
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2920
> Project: Cassandra
> Issue Type: Task
> Components: Core
> Affects Versions: 1.0
> Reporter: Jonathan Ellis
> Assignee: Stu Hood
> Priority: Minor
> Fix For: 1.0
>
> Attachments: 2920-v2.txt, 2920.txt
>
>
> CASSANDRA-2677 left Builder-oriented streaming in for backwards compatibility, but since we're going to change the sstable format for 1.0, and Builder doesn't work across format changes, there's no point in leaving Builder around.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-2920) Remove SSTableWriter.Builder
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis reassigned CASSANDRA-2920:
-----------------------------------------
Assignee: Stu Hood (was: Jonathan Ellis)
> Remove SSTableWriter.Builder
> ----------------------------
>
> Key: CASSANDRA-2920
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2920
> Project: Cassandra
> Issue Type: Task
> Components: Core
> Affects Versions: 1.0
> Reporter: Jonathan Ellis
> Assignee: Stu Hood
> Priority: Minor
> Fix For: 1.0
>
> Attachments: 2920.txt
>
>
> CASSANDRA-2677 left Builder-oriented streaming in for backwards compatibility, but since we're going to change the sstable format for 1.0, and Builder doesn't work across format changes, there's no point in leaving Builder around.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2920) Remove SSTableWriter.Builder
Posted by "Yuki Morishita (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13069540#comment-13069540 ]
Yuki Morishita commented on CASSANDRA-2920:
-------------------------------------------
OK, then +1.
> Remove SSTableWriter.Builder
> ----------------------------
>
> Key: CASSANDRA-2920
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2920
> Project: Cassandra
> Issue Type: Task
> Components: Core
> Affects Versions: 1.0
> Reporter: Jonathan Ellis
> Assignee: Stu Hood
> Priority: Minor
> Fix For: 1.0
>
> Attachments: 2920-v2.txt, 2920.txt
>
>
> CASSANDRA-2677 left Builder-oriented streaming in for backwards compatibility, but since we're going to change the sstable format for 1.0, and Builder doesn't work across format changes, there's no point in leaving Builder around.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2920) Remove SSTableWriter.Builder
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13069533#comment-13069533 ]
Jonathan Ellis commented on CASSANDRA-2920:
-------------------------------------------
It could, but the point of this ticket is that a rolling upgrade from 0.8.x to 1.0 will hit this:
{code}
if (!desc.isLatestVersion)
// TODO: streaming between different versions will fail: need support for
// recovering other versions to provide a stable streaming api
throw new RuntimeException(String.format("Cannot recover SSTable %s due to version mismatch. (current version is %s).", desc.toString()
, Descriptor.CURRENT_VERSION));
{code}
So it doesn't matter whether we support it at the streaming protocol level or not.
> Remove SSTableWriter.Builder
> ----------------------------
>
> Key: CASSANDRA-2920
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2920
> Project: Cassandra
> Issue Type: Task
> Components: Core
> Affects Versions: 1.0
> Reporter: Jonathan Ellis
> Assignee: Stu Hood
> Priority: Minor
> Fix For: 1.0
>
> Attachments: 2920-v2.txt, 2920.txt
>
>
> CASSANDRA-2677 left Builder-oriented streaming in for backwards compatibility, but since we're going to change the sstable format for 1.0, and Builder doesn't work across format changes, there's no point in leaving Builder around.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2920) Remove SSTableWriter.Builder
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13070950#comment-13070950 ]
Hudson commented on CASSANDRA-2920:
-----------------------------------
Integrated in Cassandra #973 (See [https://builds.apache.org/job/Cassandra/973/])
Remove SSTableWriter.Builder
patch by jbellis and stuhood; reviewed by Yuki Morishita for CASSANDRA-2920
jbellis : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1150984
Files :
* /cassandra/trunk/test/unit/org/apache/cassandra/io/sstable/SSTableUtils.java
* /cassandra/trunk/src/java/org/apache/cassandra/net/IncomingTcpConnection.java
* /cassandra/trunk/test/long/org/apache/cassandra/db/compaction/LongCompactionSpeedTest.java
* /cassandra/trunk/src/java/org/apache/cassandra/streaming/StreamInSession.java
* /cassandra/trunk/src/java/org/apache/cassandra/security/streaming/SSLIncomingStreamReader.java
* /cassandra/trunk/test/unit/org/apache/cassandra/streaming/StreamingTransferTest.java
* /cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
* /cassandra/trunk/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java
* /cassandra/trunk/src/java/org/apache/cassandra/streaming/IncomingStreamReader.java
> Remove SSTableWriter.Builder
> ----------------------------
>
> Key: CASSANDRA-2920
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2920
> Project: Cassandra
> Issue Type: Task
> Components: Core
> Affects Versions: 1.0
> Reporter: Jonathan Ellis
> Assignee: Stu Hood
> Priority: Minor
> Fix For: 1.0
>
> Attachments: 2920-v2.txt, 2920-v3-0001.txt, 2920-v3-0002-Fix-reference-counting-add-sstable-comparison-and-port.txt, 2920.txt
>
>
> CASSANDRA-2677 left Builder-oriented streaming in for backwards compatibility, but since we're going to change the sstable format for 1.0, and Builder doesn't work across format changes, there's no point in leaving Builder around.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2920) Remove SSTableWriter.Builder
Posted by "Yuki Morishita (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13068440#comment-13068440 ]
Yuki Morishita commented on CASSANDRA-2920:
-------------------------------------------
We don't need to check remoteFile.estimatedKeys > 0 in IncomingStreamReader no more, since CASSANDRA-2916 (maybe put an assert here?).
Also, instead of holding socketChannel, we can just hold socket.
> Remove SSTableWriter.Builder
> ----------------------------
>
> Key: CASSANDRA-2920
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2920
> Project: Cassandra
> Issue Type: Task
> Components: Core
> Affects Versions: 1.0
> Reporter: Jonathan Ellis
> Assignee: Stu Hood
> Priority: Minor
> Fix For: 1.0
>
> Attachments: 2920.txt
>
>
> CASSANDRA-2677 left Builder-oriented streaming in for backwards compatibility, but since we're going to change the sstable format for 1.0, and Builder doesn't work across format changes, there's no point in leaving Builder around.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2920) Remove SSTableWriter.Builder
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-2920:
--------------------------------------
Attachment: 2920.txt
Patch removes Builder from src, but there are a bunch of tests, mostly counter-related, that invoke Builder. It's not immediately clear to me whether this is ancillary to the test ("might as well test Builder too") or central to it.
> Remove SSTableWriter.Builder
> ----------------------------
>
> Key: CASSANDRA-2920
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2920
> Project: Cassandra
> Issue Type: Task
> Components: Core
> Affects Versions: 1.0
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Priority: Minor
> Fix For: 1.0
>
> Attachments: 2920.txt
>
>
> CASSANDRA-2677 left Builder-oriented streaming in for backwards compatibility, but since we're going to change the sstable format for 1.0, and Builder doesn't work across format changes, there's no point in leaving Builder around.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2920) Remove SSTableWriter.Builder
Posted by "Stu Hood (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13067982#comment-13067982 ]
Stu Hood commented on CASSANDRA-2920:
-------------------------------------
The paths in the tests that are using builders do so to test the builder: the codepath used in #2677 is much more straightforward, but it might be worth it to port some of the Counter tests to use streaming rather than the builder directly. I can work on this in the next few days?
> Remove SSTableWriter.Builder
> ----------------------------
>
> Key: CASSANDRA-2920
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2920
> Project: Cassandra
> Issue Type: Task
> Components: Core
> Affects Versions: 1.0
> Reporter: Jonathan Ellis
> Assignee: Stu Hood
> Priority: Minor
> Fix For: 1.0
>
> Attachments: 2920.txt
>
>
> CASSANDRA-2677 left Builder-oriented streaming in for backwards compatibility, but since we're going to change the sstable format for 1.0, and Builder doesn't work across format changes, there's no point in leaving Builder around.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira