You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Stu Hood (JIRA)" <ji...@apache.org> on 2009/11/24 20:09:40 UTC
[jira] Created: (CASSANDRA-579) Add support to io.Streaming API for
sending Streams
Add support to io.Streaming API for sending Streams
---------------------------------------------------
Key: CASSANDRA-579
URL: https://issues.apache.org/jira/browse/CASSANDRA-579
Project: Cassandra
Issue Type: Improvement
Reporter: Stu Hood
Priority: Minor
Fix For: 0.9
The io.Streaming API currently requires a file on disk to stream, which means that bootstrap and repairs need to perform an anti-compaction that writes a bunch of data to disk, only to have it be deleted after the streaming has finished.
Ideally, the Streaming API should allow for streaming from an InputStream (or any other class we think we need to design to make the streaming as efficient as possible). That way, anti-compaction for repair/bootstrap does not perform any writing: it simply streams the relevant portion of the file to the neighbor.
Additionally, this opens up interesting possibilities, such as providing the Streaming API as a (Java only?) client API. One use case would be for a Hadoop OutputFormat: rather than writing BinaryMemtables, the OutputFormat could literally write an SSTable to the stream. This might require better integration with gossip, to ensure that you aren't writing to the completely wrong node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-579) Add support to io.Streaming API
for sending Streams
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12803763#action_12803763 ]
Jonathan Ellis commented on CASSANDRA-579:
------------------------------------------
one side has to compact eventually, so it should be the sending side that compacts, since that gets you reduced bandwidth; there's no benefit by having receiver compact. it would be quite easy to compact to a socket stream instead of a disk one. (you lose transferTo, but receiver can still do transferFrom. and we could special case only-a-single-source-sstable with transferTo if that is worth it...)
> Add support to io.Streaming API for sending Streams
> ---------------------------------------------------
>
> Key: CASSANDRA-579
> URL: https://issues.apache.org/jira/browse/CASSANDRA-579
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Stu Hood
> Fix For: 0.6
>
>
> The io.Streaming API currently requires a file on disk to stream, which means that bootstrap and repairs need to perform an anti-compaction that writes a bunch of data to disk, only to have it be deleted after the streaming has finished.
> Ideally, the Streaming API should allow for streaming from an InputStream (or any other class we think we need to design to make the streaming as efficient as possible). That way, anti-compaction for repair/bootstrap does not perform any writing: it simply streams the relevant portion of the file to the neighbor.
> Additionally, this opens up interesting possibilities, such as providing the Streaming API as a (Java only?) client API. One use case would be for a Hadoop OutputFormat: rather than writing BinaryMemtables, the OutputFormat could literally write an SSTable to the stream. This might require better integration with gossip, to ensure that you aren't writing to the completely wrong node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-579) Add support to io.Streaming API
for sending Streams
Posted by "Stu Hood (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12803760#action_12803760 ]
Stu Hood commented on CASSANDRA-579:
------------------------------------
> What if we just stream from individual SSTables directly?
This idea is growing on me... it causes minimal load on the machine that you are trying to move data off of, in terms of CPU and disk (compaction is done on the destination side). Network bandwidth should be relatively plentiful.
> Add support to io.Streaming API for sending Streams
> ---------------------------------------------------
>
> Key: CASSANDRA-579
> URL: https://issues.apache.org/jira/browse/CASSANDRA-579
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Stu Hood
> Fix For: 0.6
>
>
> The io.Streaming API currently requires a file on disk to stream, which means that bootstrap and repairs need to perform an anti-compaction that writes a bunch of data to disk, only to have it be deleted after the streaming has finished.
> Ideally, the Streaming API should allow for streaming from an InputStream (or any other class we think we need to design to make the streaming as efficient as possible). That way, anti-compaction for repair/bootstrap does not perform any writing: it simply streams the relevant portion of the file to the neighbor.
> Additionally, this opens up interesting possibilities, such as providing the Streaming API as a (Java only?) client API. One use case would be for a Hadoop OutputFormat: rather than writing BinaryMemtables, the OutputFormat could literally write an SSTable to the stream. This might require better integration with gossip, to ensure that you aren't writing to the completely wrong node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-579) Add support to io.Streaming API
for sending Streams
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12791999#action_12791999 ]
Jonathan Ellis commented on CASSANDRA-579:
------------------------------------------
What you want to be careful about here is not screwing up the FileChannel.transferTo optimization which is very valuable.
So IMO what you want to do is:
1. use SSTR.getPosition to find start and end ranges to transfer, then use the existing streaming API -- which already support streaming only _parts_ of files via transferTo -- to send that over as the data file in question.
2. from the data file, compute index + BF files on the destination node, instead of wasting IO streaming those from the source.
- streaming the index from source is possible, but since you need to scan the data file anyway to build BF (since there is no way to extract a subset of a BF) I think it's going to be simpler to just rebuild both. And anyway goffinet has wanted a "rebuild index from data file" for a while now :)
> Add support to io.Streaming API for sending Streams
> ---------------------------------------------------
>
> Key: CASSANDRA-579
> URL: https://issues.apache.org/jira/browse/CASSANDRA-579
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Stu Hood
> Fix For: 0.9
>
>
> The io.Streaming API currently requires a file on disk to stream, which means that bootstrap and repairs need to perform an anti-compaction that writes a bunch of data to disk, only to have it be deleted after the streaming has finished.
> Ideally, the Streaming API should allow for streaming from an InputStream (or any other class we think we need to design to make the streaming as efficient as possible). That way, anti-compaction for repair/bootstrap does not perform any writing: it simply streams the relevant portion of the file to the neighbor.
> Additionally, this opens up interesting possibilities, such as providing the Streaming API as a (Java only?) client API. One use case would be for a Hadoop OutputFormat: rather than writing BinaryMemtables, the OutputFormat could literally write an SSTable to the stream. This might require better integration with gossip, to ensure that you aren't writing to the completely wrong node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-579) Add support to io.Streaming API for
sending Streams
Posted by "Stu Hood (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stu Hood updated CASSANDRA-579:
-------------------------------
Description:
The io.Streaming API currently requires a file on disk to stream, which means that bootstrap and repairs need to perform an anti-compaction that writes a bunch of data to disk, only to have it be deleted after the streaming has finished.
EDIT: Deleted reference to using streaming as a client API: it wouldn't provide enough benefit over using the BMT interface, due to fragility.
was:
The io.Streaming API currently requires a file on disk to stream, which means that bootstrap and repairs need to perform an anti-compaction that writes a bunch of data to disk, only to have it be deleted after the streaming has finished.
Ideally, the Streaming API should allow for streaming from an InputStream (or any other class we think we need to design to make the streaming as efficient as possible). That way, anti-compaction for repair/bootstrap does not perform any writing: it simply streams the relevant portion of the file to the neighbor.
Additionally, this opens up interesting possibilities, such as providing the Streaming API as a (Java only?) client API. One use case would be for a Hadoop OutputFormat: rather than writing BinaryMemtables, the OutputFormat could literally write an SSTable to the stream. This might require better integration with gossip, to ensure that you aren't writing to the completely wrong node.
> Add support to io.Streaming API for sending Streams
> ---------------------------------------------------
>
> Key: CASSANDRA-579
> URL: https://issues.apache.org/jira/browse/CASSANDRA-579
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Stu Hood
> Fix For: 0.7
>
>
> The io.Streaming API currently requires a file on disk to stream, which means that bootstrap and repairs need to perform an anti-compaction that writes a bunch of data to disk, only to have it be deleted after the streaming has finished.
> EDIT: Deleted reference to using streaming as a client API: it wouldn't provide enough benefit over using the BMT interface, due to fragility.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-579) Add support to io.Streaming API
for sending Streams
Posted by "Stu Hood (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12839309#action_12839309 ]
Stu Hood commented on CASSANDRA-579:
------------------------------------
We can probably resume progress here now that 389 is in, although I hope to get 777 in first.
> Add support to io.Streaming API for sending Streams
> ---------------------------------------------------
>
> Key: CASSANDRA-579
> URL: https://issues.apache.org/jira/browse/CASSANDRA-579
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Stu Hood
> Fix For: 0.7
>
>
> The io.Streaming API currently requires a file on disk to stream, which means that bootstrap and repairs need to perform an anti-compaction that writes a bunch of data to disk, only to have it be deleted after the streaming has finished.
> Ideally, the Streaming API should allow for streaming from an InputStream (or any other class we think we need to design to make the streaming as efficient as possible). That way, anti-compaction for repair/bootstrap does not perform any writing: it simply streams the relevant portion of the file to the neighbor.
> Additionally, this opens up interesting possibilities, such as providing the Streaming API as a (Java only?) client API. One use case would be for a Hadoop OutputFormat: rather than writing BinaryMemtables, the OutputFormat could literally write an SSTable to the stream. This might require better integration with gossip, to ensure that you aren't writing to the completely wrong node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-579) Add support to io.Streaming API for
sending Streams
Posted by "Stu Hood (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stu Hood updated CASSANDRA-579:
-------------------------------
Priority: Major (was: Minor)
Bumping priority on this one, because goffinet experienced a repair taking > 24 hours for ~100GB of data.
> Add support to io.Streaming API for sending Streams
> ---------------------------------------------------
>
> Key: CASSANDRA-579
> URL: https://issues.apache.org/jira/browse/CASSANDRA-579
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Stu Hood
> Fix For: 0.9
>
>
> The io.Streaming API currently requires a file on disk to stream, which means that bootstrap and repairs need to perform an anti-compaction that writes a bunch of data to disk, only to have it be deleted after the streaming has finished.
> Ideally, the Streaming API should allow for streaming from an InputStream (or any other class we think we need to design to make the streaming as efficient as possible). That way, anti-compaction for repair/bootstrap does not perform any writing: it simply streams the relevant portion of the file to the neighbor.
> Additionally, this opens up interesting possibilities, such as providing the Streaming API as a (Java only?) client API. One use case would be for a Hadoop OutputFormat: rather than writing BinaryMemtables, the OutputFormat could literally write an SSTable to the stream. This might require better integration with gossip, to ensure that you aren't writing to the completely wrong node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-579) Add support to io.Streaming API
for sending Streams
Posted by "Stu Hood (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12792034#action_12792034 ]
Stu Hood commented on CASSANDRA-579:
------------------------------------
> ...since you just did a major compact...
You didn't necessarily just do a major compaction: currently, when someone triggers a repair manually, it performs a readonly major compaction, which doesn't write to disk.
> Add support to io.Streaming API for sending Streams
> ---------------------------------------------------
>
> Key: CASSANDRA-579
> URL: https://issues.apache.org/jira/browse/CASSANDRA-579
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Stu Hood
> Fix For: 0.9
>
>
> The io.Streaming API currently requires a file on disk to stream, which means that bootstrap and repairs need to perform an anti-compaction that writes a bunch of data to disk, only to have it be deleted after the streaming has finished.
> Ideally, the Streaming API should allow for streaming from an InputStream (or any other class we think we need to design to make the streaming as efficient as possible). That way, anti-compaction for repair/bootstrap does not perform any writing: it simply streams the relevant portion of the file to the neighbor.
> Additionally, this opens up interesting possibilities, such as providing the Streaming API as a (Java only?) client API. One use case would be for a Hadoop OutputFormat: rather than writing BinaryMemtables, the OutputFormat could literally write an SSTable to the stream. This might require better integration with gossip, to ensure that you aren't writing to the completely wrong node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-579) Add support to io.Streaming API
for sending Streams
Posted by "Chris Goffinet (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12791770#action_12791770 ]
Chris Goffinet commented on CASSANDRA-579:
------------------------------------------
Also note, this was on 20 CFs
> Add support to io.Streaming API for sending Streams
> ---------------------------------------------------
>
> Key: CASSANDRA-579
> URL: https://issues.apache.org/jira/browse/CASSANDRA-579
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Stu Hood
> Fix For: 0.9
>
>
> The io.Streaming API currently requires a file on disk to stream, which means that bootstrap and repairs need to perform an anti-compaction that writes a bunch of data to disk, only to have it be deleted after the streaming has finished.
> Ideally, the Streaming API should allow for streaming from an InputStream (or any other class we think we need to design to make the streaming as efficient as possible). That way, anti-compaction for repair/bootstrap does not perform any writing: it simply streams the relevant portion of the file to the neighbor.
> Additionally, this opens up interesting possibilities, such as providing the Streaming API as a (Java only?) client API. One use case would be for a Hadoop OutputFormat: rather than writing BinaryMemtables, the OutputFormat could literally write an SSTable to the stream. This might require better integration with gossip, to ensure that you aren't writing to the completely wrong node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-579) Add support to io.Streaming API
for sending Streams
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12792030#action_12792030 ]
Jonathan Ellis commented on CASSANDRA-579:
------------------------------------------
> Unless you are suggesting streaming each SSTable individually?
streaming each range individually (so, one source sstable since you just did a major compact, and each range becomes an sstable on the target)
> Add support to io.Streaming API for sending Streams
> ---------------------------------------------------
>
> Key: CASSANDRA-579
> URL: https://issues.apache.org/jira/browse/CASSANDRA-579
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Stu Hood
> Fix For: 0.9
>
>
> The io.Streaming API currently requires a file on disk to stream, which means that bootstrap and repairs need to perform an anti-compaction that writes a bunch of data to disk, only to have it be deleted after the streaming has finished.
> Ideally, the Streaming API should allow for streaming from an InputStream (or any other class we think we need to design to make the streaming as efficient as possible). That way, anti-compaction for repair/bootstrap does not perform any writing: it simply streams the relevant portion of the file to the neighbor.
> Additionally, this opens up interesting possibilities, such as providing the Streaming API as a (Java only?) client API. One use case would be for a Hadoop OutputFormat: rather than writing BinaryMemtables, the OutputFormat could literally write an SSTable to the stream. This might require better integration with gossip, to ensure that you aren't writing to the completely wrong node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-579) Add support to io.Streaming API
for sending Streams
Posted by "Chris Goffinet (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12791771#action_12791771 ]
Chris Goffinet commented on CASSANDRA-579:
------------------------------------------
We see about 16MB/s for compaction on a 4 disk RAID0 configuration. Moving this directly to network, so we can be network bound would be much better. But the thing is, compaction is running slower than the max throughput on network, which is roughly 80MB/s on our systems.
> Add support to io.Streaming API for sending Streams
> ---------------------------------------------------
>
> Key: CASSANDRA-579
> URL: https://issues.apache.org/jira/browse/CASSANDRA-579
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Stu Hood
> Fix For: 0.9
>
>
> The io.Streaming API currently requires a file on disk to stream, which means that bootstrap and repairs need to perform an anti-compaction that writes a bunch of data to disk, only to have it be deleted after the streaming has finished.
> Ideally, the Streaming API should allow for streaming from an InputStream (or any other class we think we need to design to make the streaming as efficient as possible). That way, anti-compaction for repair/bootstrap does not perform any writing: it simply streams the relevant portion of the file to the neighbor.
> Additionally, this opens up interesting possibilities, such as providing the Streaming API as a (Java only?) client API. One use case would be for a Hadoop OutputFormat: rather than writing BinaryMemtables, the OutputFormat could literally write an SSTable to the stream. This might require better integration with gossip, to ensure that you aren't writing to the completely wrong node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-579) Add support to io.Streaming API
for sending Streams
Posted by "Stu Hood (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12829985#action_12829985 ]
Stu Hood commented on CASSANDRA-579:
------------------------------------
I think we should delay making too many changes to streaming until we've finalized the SSTable versioning/interface changes proposed on #674. All of the possible approaches to optimizing this depend on the file format. For instance, sending only portions of the file depends on a splittable format, and performing the compaction on the sending side and then writing a new SSTable on the receiving side depends on the format of the serialized CompactedRows that would be sent across.
> Add support to io.Streaming API for sending Streams
> ---------------------------------------------------
>
> Key: CASSANDRA-579
> URL: https://issues.apache.org/jira/browse/CASSANDRA-579
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Stu Hood
> Fix For: 0.7
>
>
> The io.Streaming API currently requires a file on disk to stream, which means that bootstrap and repairs need to perform an anti-compaction that writes a bunch of data to disk, only to have it be deleted after the streaming has finished.
> Ideally, the Streaming API should allow for streaming from an InputStream (or any other class we think we need to design to make the streaming as efficient as possible). That way, anti-compaction for repair/bootstrap does not perform any writing: it simply streams the relevant portion of the file to the neighbor.
> Additionally, this opens up interesting possibilities, such as providing the Streaming API as a (Java only?) client API. One use case would be for a Hadoop OutputFormat: rather than writing BinaryMemtables, the OutputFormat could literally write an SSTable to the stream. This might require better integration with gossip, to ensure that you aren't writing to the completely wrong node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-579) Add support to io.Streaming API
for sending Streams
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12786384#action_12786384 ]
Hudson commented on CASSANDRA-579:
----------------------------------
Integrated in Cassandra #278 (See [http://hudson.zones.apache.org/hudson/job/Cassandra/278/])
> Add support to io.Streaming API for sending Streams
> ---------------------------------------------------
>
> Key: CASSANDRA-579
> URL: https://issues.apache.org/jira/browse/CASSANDRA-579
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Stu Hood
> Priority: Minor
> Fix For: 0.9
>
>
> The io.Streaming API currently requires a file on disk to stream, which means that bootstrap and repairs need to perform an anti-compaction that writes a bunch of data to disk, only to have it be deleted after the streaming has finished.
> Ideally, the Streaming API should allow for streaming from an InputStream (or any other class we think we need to design to make the streaming as efficient as possible). That way, anti-compaction for repair/bootstrap does not perform any writing: it simply streams the relevant portion of the file to the neighbor.
> Additionally, this opens up interesting possibilities, such as providing the Streaming API as a (Java only?) client API. One use case would be for a Hadoop OutputFormat: rather than writing BinaryMemtables, the OutputFormat could literally write an SSTable to the stream. This might require better integration with gossip, to ensure that you aren't writing to the completely wrong node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-579) Add support to io.Streaming API
for sending Streams
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782104#action_12782104 ]
Jonathan Ellis commented on CASSANDRA-579:
------------------------------------------
expanding Streaming is fine, but it's not as straightforward as you might think because it is completely based around FileChannel.transferTo, which on Linux at least is a thin wrapper around the sendfile system call
> Add support to io.Streaming API for sending Streams
> ---------------------------------------------------
>
> Key: CASSANDRA-579
> URL: https://issues.apache.org/jira/browse/CASSANDRA-579
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Stu Hood
> Priority: Minor
> Fix For: 0.9
>
>
> The io.Streaming API currently requires a file on disk to stream, which means that bootstrap and repairs need to perform an anti-compaction that writes a bunch of data to disk, only to have it be deleted after the streaming has finished.
> Ideally, the Streaming API should allow for streaming from an InputStream (or any other class we think we need to design to make the streaming as efficient as possible). That way, anti-compaction for repair/bootstrap does not perform any writing: it simply streams the relevant portion of the file to the neighbor.
> Additionally, this opens up interesting possibilities, such as providing the Streaming API as a (Java only?) client API. One use case would be for a Hadoop OutputFormat: rather than writing BinaryMemtables, the OutputFormat could literally write an SSTable to the stream. This might require better integration with gossip, to ensure that you aren't writing to the completely wrong node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (CASSANDRA-579) Add support to
io.Streaming API for sending Streams
Posted by "Stu Hood (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12792026#action_12792026 ]
Stu Hood edited comment on CASSANDRA-579 at 12/17/09 5:53 PM:
--------------------------------------------------------------
> 1. use SSTR.getPosition to find start and end ranges to transfer...
I don't think this works: you still need to perform the compaction first, which means you still need to dump to disk.
EDIT: Unless you are suggesting streaming each SSTable individually?
> 2. from the data file, compute index + BF files on the destination node
This seems like the best approach.
was (Author: stuhood):
> 1. use SSTR.getPosition to find start and end ranges to transfer...
I don't think this works: you still need to perform the compaction first, which means you still need to dump to disk.
> 2. from the data file, compute index + BF files on the destination node
This seems like the best approach.
> Add support to io.Streaming API for sending Streams
> ---------------------------------------------------
>
> Key: CASSANDRA-579
> URL: https://issues.apache.org/jira/browse/CASSANDRA-579
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Stu Hood
> Fix For: 0.9
>
>
> The io.Streaming API currently requires a file on disk to stream, which means that bootstrap and repairs need to perform an anti-compaction that writes a bunch of data to disk, only to have it be deleted after the streaming has finished.
> Ideally, the Streaming API should allow for streaming from an InputStream (or any other class we think we need to design to make the streaming as efficient as possible). That way, anti-compaction for repair/bootstrap does not perform any writing: it simply streams the relevant portion of the file to the neighbor.
> Additionally, this opens up interesting possibilities, such as providing the Streaming API as a (Java only?) client API. One use case would be for a Hadoop OutputFormat: rather than writing BinaryMemtables, the OutputFormat could literally write an SSTable to the stream. This might require better integration with gossip, to ensure that you aren't writing to the completely wrong node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-579) Add support to io.Streaming API
for sending Streams
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12806558#action_12806558 ]
Jonathan Ellis commented on CASSANDRA-579:
------------------------------------------
Note that if we're streaming w/o first writing temporary file locally there is less reason to keep all outbound streaming single-threaded, but Streaming contexts currently assume there can only be one outstanding stream per source node. So keep that in mind if we are tempted to multithread streaming.
> Add support to io.Streaming API for sending Streams
> ---------------------------------------------------
>
> Key: CASSANDRA-579
> URL: https://issues.apache.org/jira/browse/CASSANDRA-579
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Stu Hood
> Fix For: 0.7
>
>
> The io.Streaming API currently requires a file on disk to stream, which means that bootstrap and repairs need to perform an anti-compaction that writes a bunch of data to disk, only to have it be deleted after the streaming has finished.
> Ideally, the Streaming API should allow for streaming from an InputStream (or any other class we think we need to design to make the streaming as efficient as possible). That way, anti-compaction for repair/bootstrap does not perform any writing: it simply streams the relevant portion of the file to the neighbor.
> Additionally, this opens up interesting possibilities, such as providing the Streaming API as a (Java only?) client API. One use case would be for a Hadoop OutputFormat: rather than writing BinaryMemtables, the OutputFormat could literally write an SSTable to the stream. This might require better integration with gossip, to ensure that you aren't writing to the completely wrong node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-579) Add support to io.Streaming API
for sending Streams
Posted by "Stu Hood (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12792026#action_12792026 ]
Stu Hood commented on CASSANDRA-579:
------------------------------------
> 1. use SSTR.getPosition to find start and end ranges to transfer...
I don't think this works: you still need to perform the compaction first, which means you still need to dump to disk.
> 2. from the data file, compute index + BF files on the destination node
This seems like the best approach.
> Add support to io.Streaming API for sending Streams
> ---------------------------------------------------
>
> Key: CASSANDRA-579
> URL: https://issues.apache.org/jira/browse/CASSANDRA-579
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Stu Hood
> Fix For: 0.9
>
>
> The io.Streaming API currently requires a file on disk to stream, which means that bootstrap and repairs need to perform an anti-compaction that writes a bunch of data to disk, only to have it be deleted after the streaming has finished.
> Ideally, the Streaming API should allow for streaming from an InputStream (or any other class we think we need to design to make the streaming as efficient as possible). That way, anti-compaction for repair/bootstrap does not perform any writing: it simply streams the relevant portion of the file to the neighbor.
> Additionally, this opens up interesting possibilities, such as providing the Streaming API as a (Java only?) client API. One use case would be for a Hadoop OutputFormat: rather than writing BinaryMemtables, the OutputFormat could literally write an SSTable to the stream. This might require better integration with gossip, to ensure that you aren't writing to the completely wrong node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-579) Add support to io.Streaming API for
sending Streams
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-579:
-------------------------------------
Fix Version/s: (was: 0.6)
0.7
> Add support to io.Streaming API for sending Streams
> ---------------------------------------------------
>
> Key: CASSANDRA-579
> URL: https://issues.apache.org/jira/browse/CASSANDRA-579
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Stu Hood
> Fix For: 0.7
>
>
> The io.Streaming API currently requires a file on disk to stream, which means that bootstrap and repairs need to perform an anti-compaction that writes a bunch of data to disk, only to have it be deleted after the streaming has finished.
> Ideally, the Streaming API should allow for streaming from an InputStream (or any other class we think we need to design to make the streaming as efficient as possible). That way, anti-compaction for repair/bootstrap does not perform any writing: it simply streams the relevant portion of the file to the neighbor.
> Additionally, this opens up interesting possibilities, such as providing the Streaming API as a (Java only?) client API. One use case would be for a Hadoop OutputFormat: rather than writing BinaryMemtables, the OutputFormat could literally write an SSTable to the stream. This might require better integration with gossip, to ensure that you aren't writing to the completely wrong node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-579) Add support to io.Streaming API
for sending Streams
Posted by "Jaakko Laine (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12802566#action_12802566 ]
Jaakko Laine commented on CASSANDRA-579:
----------------------------------------
What if we just stream from individual SSTables directly? That would of course send some stuff twice (or more) if done without compaction, but that seems like small overhead. If we streamed directly from SSTables, that would allow us to bootstrap from a node that is short of disk space.
> Add support to io.Streaming API for sending Streams
> ---------------------------------------------------
>
> Key: CASSANDRA-579
> URL: https://issues.apache.org/jira/browse/CASSANDRA-579
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Stu Hood
> Fix For: 0.6
>
>
> The io.Streaming API currently requires a file on disk to stream, which means that bootstrap and repairs need to perform an anti-compaction that writes a bunch of data to disk, only to have it be deleted after the streaming has finished.
> Ideally, the Streaming API should allow for streaming from an InputStream (or any other class we think we need to design to make the streaming as efficient as possible). That way, anti-compaction for repair/bootstrap does not perform any writing: it simply streams the relevant portion of the file to the neighbor.
> Additionally, this opens up interesting possibilities, such as providing the Streaming API as a (Java only?) client API. One use case would be for a Hadoop OutputFormat: rather than writing BinaryMemtables, the OutputFormat could literally write an SSTable to the stream. This might require better integration with gossip, to ensure that you aren't writing to the completely wrong node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.