You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by JR <ma...@gmail.com> on 2013/04/01 03:56:44 UTC
How do I search the past posts for a topic?
Hello,
I would like to search past posts on a topic like "netcat" or "logger".
When I have a 'netcat' source and sink as "logger" - what should I do to
produce an event and how will the logger get any data?
I provided logger as "console".
My cmd:
$FLUME_HOME/bin/flume-ng agent --conf-file flumeAgent_netCat2logger.conf
--conf $FLUME_HOME/conf --name a1 -Dflume.root.logger=INFO,console
-------------------------------------------------
Conf:
a1.sources = r1
a1.sinks = k1
a1.channels = c1
# Describe/configure the source
a1.sources.r1.type = netcat
a1.sources.r1.bind = localhost
a1.sources.r1.port = 44444
# Bind the source and sink to the channel
a1.sources.r1.channels = c1
# Describe the sink
a1.sinks.k1.type = logger
# Use a channel which buffers events in memory
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100
# Bind the source and sink to the channel
a1.sinks.k1.channel = c1
~
Thanks!
Jayashree
Re: How do I search the past posts for a topic?
Posted by JR <ma...@gmail.com>.
Thanks everyone.
On Thu, Apr 4, 2013 at 3:32 PM, Otis Gospodnetic <otis_gospodnetic@yahoo.com
> wrote:
> Hi,
>
> Or if you prefer slightly prettier URLs: http://search-hadoop.com/Flume
>
> And if you start your query with "flume", you automatically limit your
> query to the Flume project, like this:
> http://www.search-hadoop.com/?q=flume+mike
>
> Otis
> ----
> Performance Monitoring for Solr / ElasticSearch / HBase -
> http://sematext.com/spm
>
>
>
> ------------------------------
> *From:* Mike Percy <mp...@apache.org>
> *To:* "user@flume.apache.org" <us...@flume.apache.org>
> *Sent:* Wednesday, April 3, 2013 10:41 PM
> *Subject:* Re: How do I search the past posts for a topic?
>
> Jayashree,
> I like to use the search-hadoop.com site provided by Sematext:
> http://search-hadoop.com/?q=&fc_project=Flume
>
> The logger is intended mainly for debugging. It will print data to the
> flume.log file itself.
>
> Regards,
> Mike
>
>
>
> On Sun, Mar 31, 2013 at 6:56 PM, JR <ma...@gmail.com> wrote:
>
> Hello,
>
> I would like to search past posts on a topic like "netcat" or "logger".
>
> When I have a 'netcat' source and sink as "logger" - what should I do to
> produce an event and how will the logger get any data?
>
> I provided logger as "console".
>
> My cmd:
> $FLUME_HOME/bin/flume-ng agent --conf-file flumeAgent_netCat2logger.conf
> --conf $FLUME_HOME/conf --name a1 -Dflume.root.logger=INFO,console
>
> -------------------------------------------------
>
> Conf:
> a1.sources = r1
> a1.sinks = k1
> a1.channels = c1
>
> # Describe/configure the source
> a1.sources.r1.type = netcat
> a1.sources.r1.bind = localhost
> a1.sources.r1.port = 44444
> # Bind the source and sink to the channel
> a1.sources.r1.channels = c1
>
> # Describe the sink
> a1.sinks.k1.type = logger
>
> # Use a channel which buffers events in memory
> a1.channels.c1.type = memory
> a1.channels.c1.capacity = 1000
> a1.channels.c1.transactionCapacity = 100
> # Bind the source and sink to the channel
> a1.sinks.k1.channel = c1
> ~
>
> Thanks!
> Jayashree
>
>
>
>
>
>
>
>
Re: How do I search the past posts for a topic?
Posted by Otis Gospodnetic <ot...@yahoo.com>.
Hi,
Or if you prefer slightly prettier URLs: http://search-hadoop.com/Flume
And if you start your query with "flume", you automatically limit your query to the Flume project, like this: http://www.search-hadoop.com/?q=flume+mike
Otis
----
Performance Monitoring for Solr / ElasticSearch / HBase - http://sematext.com/spm
>________________________________
> From: Mike Percy <mp...@apache.org>
>To: "user@flume.apache.org" <us...@flume.apache.org>
>Sent: Wednesday, April 3, 2013 10:41 PM
>Subject: Re: How do I search the past posts for a topic?
>
>
>Jayashree,
>I like to use the search-hadoop.com site provided by Sematext: http://search-hadoop.com/?q=&fc_project=Flume
>
>
>The logger is intended mainly for debugging. It will print data to the flume.log file itself.
>
>
>Regards,
>Mike
>
>
>
>
>
>On Sun, Mar 31, 2013 at 6:56 PM, JR <ma...@gmail.com> wrote:
>
>Hello,
>>
>> I would like to search past posts on a topic like "netcat" or "logger".
>>
>>When I have a 'netcat' source and sink as "logger" - what should I do to produce an event and how will the logger get any data?
>>
>>I provided logger as "console".
>>
>>My cmd:
>> $FLUME_HOME/bin/flume-ng agent --conf-file flumeAgent_netCat2logger.conf --conf $FLUME_HOME/conf --name a1 -Dflume.root.logger=INFO,console
>>
>>-------------------------------------------------
>>
>>Conf:
>>a1.sources = r1
>>a1.sinks = k1
>>a1.channels = c1
>>
>># Describe/configure the source
>>a1.sources.r1.type = netcat
>>a1.sources.r1.bind = localhost
>>a1.sources.r1.port = 44444
>># Bind the source and sink to the channel
>>a1.sources.r1.channels = c1
>>
>># Describe the sink
>>a1.sinks.k1.type = logger
>>
>># Use a channel which buffers events in memory
>>a1.channels.c1.type = memory
>>a1.channels.c1.capacity = 1000
>>a1.channels.c1.transactionCapacity = 100
>># Bind the source and sink to the channel
>>a1.sinks.k1.channel = c1
>>~
>>
>>Thanks!
>>Jayashree
>>
>>
>>
>>
>>
>>
>>
>
>
>
Re: How do I search the past posts for a topic?
Posted by Mike Percy <mp...@apache.org>.
Jayashree,
I like to use the search-hadoop.com site provided by Sematext:
http://search-hadoop.com/?q=&fc_project=Flume
The logger is intended mainly for debugging. It will print data to the
flume.log file itself.
Regards,
Mike
On Sun, Mar 31, 2013 at 6:56 PM, JR <ma...@gmail.com> wrote:
> Hello,
>
> I would like to search past posts on a topic like "netcat" or "logger".
>
> When I have a 'netcat' source and sink as "logger" - what should I do to
> produce an event and how will the logger get any data?
>
> I provided logger as "console".
>
> My cmd:
> $FLUME_HOME/bin/flume-ng agent --conf-file flumeAgent_netCat2logger.conf
> --conf $FLUME_HOME/conf --name a1 -Dflume.root.logger=INFO,console
>
> -------------------------------------------------
>
> Conf:
> a1.sources = r1
> a1.sinks = k1
> a1.channels = c1
>
> # Describe/configure the source
> a1.sources.r1.type = netcat
> a1.sources.r1.bind = localhost
> a1.sources.r1.port = 44444
> # Bind the source and sink to the channel
> a1.sources.r1.channels = c1
>
> # Describe the sink
> a1.sinks.k1.type = logger
>
> # Use a channel which buffers events in memory
> a1.channels.c1.type = memory
> a1.channels.c1.capacity = 1000
> a1.channels.c1.transactionCapacity = 100
> # Bind the source and sink to the channel
> a1.sinks.k1.channel = c1
> ~
>
> Thanks!
> Jayashree
>
>
>
>
>