RE: Moving from Flume OG to Flume NG

["Kumar, Deepak8 " <de...@citi.com>]

Hi Shumin,
I have gone through the documentation, but it don't have much. Do you have any use case in which you are using CustomSource? As in flume 0.9.4, we do have flume master url where we can see the source/sinks/decorators etc. How we can verify here whether our custom source is fine & up or have any errors?

It would be much help if you please guide me a bit.

Regards,
Deepak

From: Shumin Wu [mailto:shumin.wu@gmail.com]
Sent: Tuesday, September 18, 2012 2:11 PM
To: user@flume.apache.org
Subject: Re: Moving from Flume OG to Flume NG

Hi Deepark,

I had hard time switching from OG to NG in the beginning, too. The new way in NG to make your custom source would look like this:

public class CustomSource extends AbstractSource implements PollableSource, Configurable

or

public class CustomSource extends AbstractSource implements EventDrivenSource, Configurable

, depending on your source type.

Brock's link is a good reading resource.

Shumin
On Tue, Sep 18, 2012 at 11:05 AM, Brock Noland <br...@cloudera.com>> wrote:
Hi,

The flume developer guide as these details:

http://flume.apache.org/FlumeDeveloperGuide.html

Brock

On Tue, Sep 18, 2012 at 1:02 PM, Kumar, Deepak8 <de...@citi.com>> wrote:
> Hi,
> I am trying to switch from Flume 0.94 to Flume NG. In previous version we do have the static class EventSource.Base through which we can create our own Source or Sink. I don't find anything as such here in flume NG.
>
> Could you please guide me how do we create our own Sources here in Flume NG?
>
> Regards,
> Deepak


--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/

Re: Moving from Flume OG to Flume NG

[Shumin Wu <sh...@gmail.com>]

Our company has some BI that of course cannot be available in the default
sources provided by flume-ng. So yes, I am using the custom source, in a
way that is pretty much like the code snippet I have pasted in last email.

So far I have found the flume log a good resource to scrub error/exception
informations. You may configure the log4j property file to dump any many
details as you can.

AFAIK, there is no master/node notion in NG any longer. If you want to set
up a distributed cluster, you may take a look at the "setting multi-agent
flow" at
https://people.apache.org/~mpercy/flume/flume-1.2.0-incubating-SNAPSHOT/docs/FlumeUserGuide.html

Good luck!

Shumin

On Wed, Oct 3, 2012 at 3:42 AM, Kumar, Deepak8 <de...@citi.com>wrote:

> Hi Shumin,****
>
> I have gone through the documentation, but it don’t have much. Do you have
> any use case in which you are using CustomSource? As in flume 0.9.4, we do
> have flume master url where we can see the source/sinks/decorators etc. How
> we can verify here whether our custom source is fine & up or have any
> errors?****
>
> ** **
>
> It would be much help if you please guide me a bit.****
>
> ** **
>
> Regards,****
>
> Deepak****
>
> ** **
>
> *From:* Shumin Wu [mailto:shumin.wu@gmail.com]
> *Sent:* Tuesday, September 18, 2012 2:11 PM
> *To:* user@flume.apache.org
> *Subject:* Re: Moving from Flume OG to Flume NG****
>
> ** **
>
> Hi Deepark,
>
> I had hard time switching from OG to NG in the beginning, too. The new way
> in NG to make your custom source would look like this:
>
> public class CustomSource extends AbstractSource implements
> PollableSource, Configurable
>
> or
>
> public class CustomSource extends AbstractSource implements
> EventDrivenSource, Configurable
>
> , depending on your source type.
>
> Brock's link is a good reading resource.
>
> Shumin****
>
> On Tue, Sep 18, 2012 at 11:05 AM, Brock Noland <br...@cloudera.com> wrote:
> ****
>
> Hi,
>
> The flume developer guide as these details:
>
> http://flume.apache.org/FlumeDeveloperGuide.html
>
> Brock****
>
>
> On Tue, Sep 18, 2012 at 1:02 PM, Kumar, Deepak8 <de...@citi.com>
> wrote:
> > Hi,
> > I am trying to switch from Flume 0.94 to Flume NG. In previous version
> we do have the static class EventSource.Base through which we can create
> our own Source or Sink. I don't find anything as such here in flume NG.
> >
> > Could you please guide me how do we create our own Sources here in Flume
> NG?
> >
> > Regards,
> > Deepak
>
>
> ****
>
> --
> Apache MRUnit - Unit testing MapReduce -
> http://incubator.apache.org/mrunit/****
>
> ** **
>