You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "Praveen Ramachandra (Created) (JIRA)" <ji...@apache.org> on 2011/11/22 12:32:40 UTC
[jira] [Created] (FLUME-856) Move data across hosts
Move data across hosts
----------------------
Key: FLUME-856
URL: https://issues.apache.org/jira/browse/FLUME-856
Project: Flume
Issue Type: Question
Components: Channel, Configuration, Node, Sinks+Sources
Affects Versions: NG alpha 1
Reporter: Praveen Ramachandra
Priority: Minor
I have a custom EventDriven source that receives events over the net and needs to propagate events to multiple sinks on two different hosts in different data centers.
How do one configure the data path from source-node to destination across hosts.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-856) Move data across hosts
Posted by "Praveen Ramachandra (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13171103#comment-13171103 ]
Praveen Ramachandra commented on FLUME-856:
-------------------------------------------
I think this works. Thanks.
> Move data across hosts
> ----------------------
>
> Key: FLUME-856
> URL: https://issues.apache.org/jira/browse/FLUME-856
> Project: Flume
> Issue Type: Question
> Components: Channel, Configuration, Node, Sinks+Sources
> Affects Versions: NG alpha 1
> Reporter: Praveen Ramachandra
> Assignee: E. Sammer
> Priority: Minor
> Labels: NG, newbie
>
> I have a custom EventDriven source that receives events over the net and needs to propagate events to multiple sinks on two different hosts in different data centers.
> How do one configure the data path from source-node to destination across hosts.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-856) Move data across hosts
Posted by "E. Sammer (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162266#comment-13162266 ]
E. Sammer commented on FLUME-856:
---------------------------------
Praveen:
The proper way to get events across hosts is to use the AvroSource and AvroSink. The AvroSink is just an Avro client that knows how to send events to the an AvroSource. Here's an example of two machines (A and B) and sending data from A to B.
A.sources = seq1
A.channels = mem1
A.sinks = avroSink
B.sources = avroSource
B.channels = mem2
B.sinks = log1
A.sources.seq.type = seq
A.sources.seq.channels = mem1
A.channels.mem1.type = memory
A.sinks.avroSink.type = avro
A.sinks.avroSink.hostname = B
A.sinks.avroSink.port = 12345
A.sinks.avroSink.channel = mem1
B.sources.avroSource.type = avro
B.sources.avroSource.bind = 0.0.0.0
B.sources.avroSource.port = 12345
B.sources.seq.channels = mem2
B.channels.mem1.type = memory
B.sinks.log1.type = logger
B.sinks.log1.channel = mem1
The resulting dataflow is:
A:
seq -> memory channel -> avro client
B:
avro server -> memory channel -> logger
Let me know if it makes sense to close this.
Thanks!
> Move data across hosts
> ----------------------
>
> Key: FLUME-856
> URL: https://issues.apache.org/jira/browse/FLUME-856
> Project: Flume
> Issue Type: Question
> Components: Channel, Configuration, Node, Sinks+Sources
> Affects Versions: NG alpha 1
> Reporter: Praveen Ramachandra
> Priority: Minor
> Labels: NG, newbie
>
> I have a custom EventDriven source that receives events over the net and needs to propagate events to multiple sinks on two different hosts in different data centers.
> How do one configure the data path from source-node to destination across hosts.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (FLUME-856) Move data across hosts
Posted by "E. Sammer (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
E. Sammer resolved FLUME-856.
-----------------------------
Resolution: Fixed
Fix Version/s: v1.0.0
Resolving for now. If the AvroSource / AvroSink pair doesn't work, let's reopen this.
> Move data across hosts
> ----------------------
>
> Key: FLUME-856
> URL: https://issues.apache.org/jira/browse/FLUME-856
> Project: Flume
> Issue Type: Question
> Components: Channel, Configuration, Node, Sinks+Sources
> Affects Versions: NG alpha 1
> Reporter: Praveen Ramachandra
> Assignee: E. Sammer
> Priority: Minor
> Labels: NG, newbie
> Fix For: v1.0.0
>
>
> I have a custom EventDriven source that receives events over the net and needs to propagate events to multiple sinks on two different hosts in different data centers.
> How do one configure the data path from source-node to destination across hosts.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (FLUME-856) Move data across hosts
Posted by "E. Sammer (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
E. Sammer reassigned FLUME-856:
-------------------------------
Assignee: E. Sammer
> Move data across hosts
> ----------------------
>
> Key: FLUME-856
> URL: https://issues.apache.org/jira/browse/FLUME-856
> Project: Flume
> Issue Type: Question
> Components: Channel, Configuration, Node, Sinks+Sources
> Affects Versions: NG alpha 1
> Reporter: Praveen Ramachandra
> Assignee: E. Sammer
> Priority: Minor
> Labels: NG, newbie
>
> I have a custom EventDriven source that receives events over the net and needs to propagate events to multiple sinks on two different hosts in different data centers.
> How do one configure the data path from source-node to destination across hosts.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira