You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by Sharninder <sh...@gmail.com> on 2014/07/21 14:26:30 UTC
Custom sink/source
Hey,
I'm trying to write a new custom source/sink for flume and was going
through the source of some of the existing sinks. Essentially to write a
sink, I have to implement a start(), stop() and process().
Now, my question is if I initiate a connection in start(), send an event in
process, will I be opening another connection again for the second event?
When is stop() called?
Has anyone implemented a websocket sink that I could take a look at?
--
Sharninder
Re: Custom sink/source
Posted by Jeff Lord <jl...@cloudera.com>.
We should fix the docs
Would you be able to create a jira for that?
On Monday, July 21, 2014, Sharninder <sh...@gmail.com> wrote:
> I did look at the developer guide and was a little confused since there
> seem to be a couple of omissions in it. In particular, the even based
> source isn't explained very well. I'm new to Java (and flume) so maybe
> that's just me.
>
> Your explanation helps.
>
> --
> Sharninder
>
> On 21-Jul-2014, at 8:33 pm, Jeff Lord <jlord@cloudera.com
> <javascript:_e(%7B%7D,'cvml','jlord@cloudera.com');>> wrote:
>
> start() is called when the agent is started and the sink component is then
> started.
> calling process() will take a batch of events off the channel and send to
> the next hop or terminal location.
> stop() is called when the agent is shutdown and the sink component
> resources are unloaded.
> Have you seen the developer guide?
> https://flume.apache.org/FlumeDeveloperGuide.html#sink
>
>
> On Mon, Jul 21, 2014 at 5:26 AM, Sharninder <sharninder@gmail.com
> <javascript:_e(%7B%7D,'cvml','sharninder@gmail.com');>> wrote:
>
>> Hey,
>>
>> I'm trying to write a new custom source/sink for flume and was going
>> through the source of some of the existing sinks. Essentially to write a
>> sink, I have to implement a start(), stop() and process().
>>
>> Now, my question is if I initiate a connection in start(), send an event
>> in process, will I be opening another connection again for the second
>> event? When is stop() called?
>>
>> Has anyone implemented a websocket sink that I could take a look at?
>>
>> --
>> Sharninder
>>
>>
>>
>>
>>
>
Re: Custom sink/source
Posted by Sharninder <sh...@gmail.com>.
I did look at the developer guide and was a little confused since there seem to be a couple of omissions in it. In particular, the even based source isn't explained very well. I'm new to Java (and flume) so maybe that's just me.
Your explanation helps.
--
Sharninder
> On 21-Jul-2014, at 8:33 pm, Jeff Lord <jl...@cloudera.com> wrote:
>
> start() is called when the agent is started and the sink component is then started.
> calling process() will take a batch of events off the channel and send to the next hop or terminal location.
> stop() is called when the agent is shutdown and the sink component resources are unloaded.
> Have you seen the developer guide?
> https://flume.apache.org/FlumeDeveloperGuide.html#sink
>
>
>> On Mon, Jul 21, 2014 at 5:26 AM, Sharninder <sh...@gmail.com> wrote:
>> Hey,
>>
>> I'm trying to write a new custom source/sink for flume and was going through the source of some of the existing sinks. Essentially to write a sink, I have to implement a start(), stop() and process().
>>
>> Now, my question is if I initiate a connection in start(), send an event in process, will I be opening another connection again for the second event? When is stop() called?
>>
>> Has anyone implemented a websocket sink that I could take a look at?
>>
>> --
>> Sharninder
>
Re: Custom sink/source
Posted by Jeff Lord <jl...@cloudera.com>.
start() is called when the agent is started and the sink component is then
started.
calling process() will take a batch of events off the channel and send to
the next hop or terminal location.
stop() is called when the agent is shutdown and the sink component
resources are unloaded.
Have you seen the developer guide?
https://flume.apache.org/FlumeDeveloperGuide.html#sink
On Mon, Jul 21, 2014 at 5:26 AM, Sharninder <sh...@gmail.com> wrote:
> Hey,
>
> I'm trying to write a new custom source/sink for flume and was going
> through the source of some of the existing sinks. Essentially to write a
> sink, I have to implement a start(), stop() and process().
>
> Now, my question is if I initiate a connection in start(), send an event
> in process, will I be opening another connection again for the second
> event? When is stop() called?
>
> Has anyone implemented a websocket sink that I could take a look at?
>
> --
> Sharninder
>
>
>
>
>