You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by Anthony Biacco <ab...@handll.com> on 2014/10/16 22:38:59 UTC
flume with gnip
We were using flume with twitter's sample firehose but now would like to
use it with a GNIP hose we have.
Does anybody know how we would configure a source for this? I would assume
we'd have to use the gnip4j jar, but I haven't figured out anything beyond
that.
A google search didn't produce any help or examples.
Much appreciated,
-Tony
Re: flume with gnip
Posted by Anthony Biacco <ab...@handll.com>.
Ok, thanks. We're still learning using flume and appreciate the nudge in
the right direction.
-Tony
On Fri, Oct 17, 2014 at 12:50 AM, Ashish <pa...@gmail.com> wrote:
> Hi Anthony,
>
> Had a quick look at gnip4j. You need to write a Source for that, and
> it would be inline with Twitter source. Code would be something like
> (pulled from example), rest would be similar to Twitter Source
>
> final StreamNotification observer = new StreamNotificationAdapter() {
> @Override
> public void notify(final Activity activity, final GnipStream stream) {
> // Create Event out of Activity
>
> // get Activity as byte array
>
> Event event = EventBuilder.withBody(bytes);
> getChannelProcessor().processEvent(event);
>
> }
> };
>
> HTH !
> ashish
>
> On Fri, Oct 17, 2014 at 2:08 AM, Anthony Biacco <ab...@handll.com>
> wrote:
> >
> > We were using flume with twitter's sample firehose but now would like to
> use
> > it with a GNIP hose we have.
> >
> > Does anybody know how we would configure a source for this? I would
> assume
> > we'd have to use the gnip4j jar, but I haven't figured out anything
> beyond
> > that.
> > A google search didn't produce any help or examples.
> >
> > Much appreciated,
> >
> > -Tony
> >
>
>
>
> --
> thanks
> ashish
>
> Blog: http://www.ashishpaliwal.com/blog
> My Photo Galleries: http://www.pbase.com/ashishpaliwal
>
Re: flume with gnip
Posted by Ashish <pa...@gmail.com>.
Hi Anthony,
Had a quick look at gnip4j. You need to write a Source for that, and
it would be inline with Twitter source. Code would be something like
(pulled from example), rest would be similar to Twitter Source
final StreamNotification observer = new StreamNotificationAdapter() {
@Override
public void notify(final Activity activity, final GnipStream stream) {
// Create Event out of Activity
// get Activity as byte array
Event event = EventBuilder.withBody(bytes);
getChannelProcessor().processEvent(event);
}
};
HTH !
ashish
On Fri, Oct 17, 2014 at 2:08 AM, Anthony Biacco <ab...@handll.com> wrote:
>
> We were using flume with twitter's sample firehose but now would like to use
> it with a GNIP hose we have.
>
> Does anybody know how we would configure a source for this? I would assume
> we'd have to use the gnip4j jar, but I haven't figured out anything beyond
> that.
> A google search didn't produce any help or examples.
>
> Much appreciated,
>
> -Tony
>
--
thanks
ashish
Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal
Re: flume with gnip
Posted by Sharninder <sh...@gmail.com>.
yes a source. My bad :)
On Fri, Oct 17, 2014 at 9:54 PM, Anthony Biacco <ab...@handll.com> wrote:
> You mean a custom source, right? Not a sink.
>
> -Tony
>
>
> On Thu, Oct 16, 2014 at 10:47 PM, Sharninder <sh...@gmail.com> wrote:
>
>> You need to write a custom sink using the flume sdk. I'm not sure what
>> are the APIs provided by the gnip4 jar but it shouldn't be too difficult to
>> write the flume part.
>>
>> --
>> Sharninder
>>
>>
>> On Fri, Oct 17, 2014 at 2:08 AM, Anthony Biacco <ab...@handll.com>
>> wrote:
>>
>>>
>>> We were using flume with twitter's sample firehose but now would like to
>>> use it with a GNIP hose we have.
>>>
>>> Does anybody know how we would configure a source for this? I would
>>> assume we'd have to use the gnip4j jar, but I haven't figured out anything
>>> beyond that.
>>> A google search didn't produce any help or examples.
>>>
>>> Much appreciated,
>>>
>>> -Tony
>>>
>>>
>>
>
Re: flume with gnip
Posted by Anthony Biacco <ab...@handll.com>.
You mean a custom source, right? Not a sink.
-Tony
On Thu, Oct 16, 2014 at 10:47 PM, Sharninder <sh...@gmail.com> wrote:
> You need to write a custom sink using the flume sdk. I'm not sure what are
> the APIs provided by the gnip4 jar but it shouldn't be too difficult to
> write the flume part.
>
> --
> Sharninder
>
>
> On Fri, Oct 17, 2014 at 2:08 AM, Anthony Biacco <ab...@handll.com>
> wrote:
>
>>
>> We were using flume with twitter's sample firehose but now would like to
>> use it with a GNIP hose we have.
>>
>> Does anybody know how we would configure a source for this? I would
>> assume we'd have to use the gnip4j jar, but I haven't figured out anything
>> beyond that.
>> A google search didn't produce any help or examples.
>>
>> Much appreciated,
>>
>> -Tony
>>
>>
>
Re: flume with gnip
Posted by Sharninder <sh...@gmail.com>.
You need to write a custom sink using the flume sdk. I'm not sure what are
the APIs provided by the gnip4 jar but it shouldn't be too difficult to
write the flume part.
--
Sharninder
On Fri, Oct 17, 2014 at 2:08 AM, Anthony Biacco <ab...@handll.com> wrote:
>
> We were using flume with twitter's sample firehose but now would like to
> use it with a GNIP hose we have.
>
> Does anybody know how we would configure a source for this? I would assume
> we'd have to use the gnip4j jar, but I haven't figured out anything beyond
> that.
> A google search didn't produce any help or examples.
>
> Much appreciated,
>
> -Tony
>
>