You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by Israel Ekpo <is...@aicer.org> on 2013/04/20 19:43:25 UTC
[NEW FEATURE] - FLUME-1687 - Solr Sink for Apache Flume Now In Beta
Fellow Flume Users,
I have just created an Apache Solr sink for Flume against version 1.3.1
This has been tested and it works fine.
This sink is a great alternative to the ElasticSearchSink.
There are some flume users that have experience with Apache Solr but do not
necessarily understand how to get ElasticSearch up and running.
Having a SolrSink as an alternative could be very helpful in creating a
user interface for searching through event and log data collected with
Flume using Apache Solr.
This sink essentially picks up events from the channel and uses the
ConcurrentUpdateSolrServer client from SolrJ to send events in batches as
SolrInputDocuments using multipe worker threads to Apache Solr.
The number of documents/events per batch and the number of worker threads
can be configured.
There is a default serializer as well.
If you have the chance please try it out and let me know what you think.
It would be great to gather some feedback from users before finalizing the
features and configuration options for this sink.
Instructions are available here on how to set up and configure this sink.
https://cwiki.apache.org/confluence/display/FLUME/How+to+Setup+Solr+Sink+for+Flume
JAR files for the feature as well as its dependencies are available here
https://issues.apache.org/jira/browse/FLUME-1687
Re: [NEW FEATURE] - FLUME-1687 - Solr Sink for Apache Flume Now In Beta
Posted by Mike Percy <mp...@apache.org>.
Israel,
Nice! I'll find some time to dig into your patch this week.
Regards,
Mike
On Sat, Apr 20, 2013 at 10:43 AM, Israel Ekpo <is...@aicer.org> wrote:
> Fellow Flume Users,
>
> I have just created an Apache Solr sink for Flume against version 1.3.1
>
> This has been tested and it works fine.
>
> This sink is a great alternative to the ElasticSearchSink.
>
> There are some flume users that have experience with Apache Solr but do not
> necessarily understand how to get ElasticSearch up and running.
>
> Having a SolrSink as an alternative could be very helpful in creating a
> user interface for searching through event and log data collected with
> Flume using Apache Solr.
>
> This sink essentially picks up events from the channel and uses the
> ConcurrentUpdateSolrServer client from SolrJ to send events in batches as
> SolrInputDocuments using multipe worker threads to Apache Solr.
>
> The number of documents/events per batch and the number of worker threads
> can be configured.
>
> There is a default serializer as well.
>
> If you have the chance please try it out and let me know what you think.
>
> It would be great to gather some feedback from users before finalizing the
> features and configuration options for this sink.
>
> Instructions are available here on how to set up and configure this sink.
>
>
> https://cwiki.apache.org/confluence/display/FLUME/How+to+Setup+Solr+Sink+for+Flume
>
> JAR files for the feature as well as its dependencies are available here
>
> https://issues.apache.org/jira/browse/FLUME-1687
>
Re: [NEW FEATURE] - FLUME-1687 - Solr Sink for Apache Flume Now In Beta
Posted by Mike Percy <mp...@apache.org>.
Israel,
Nice! I'll find some time to dig into your patch this week.
Regards,
Mike
On Sat, Apr 20, 2013 at 10:43 AM, Israel Ekpo <is...@aicer.org> wrote:
> Fellow Flume Users,
>
> I have just created an Apache Solr sink for Flume against version 1.3.1
>
> This has been tested and it works fine.
>
> This sink is a great alternative to the ElasticSearchSink.
>
> There are some flume users that have experience with Apache Solr but do not
> necessarily understand how to get ElasticSearch up and running.
>
> Having a SolrSink as an alternative could be very helpful in creating a
> user interface for searching through event and log data collected with
> Flume using Apache Solr.
>
> This sink essentially picks up events from the channel and uses the
> ConcurrentUpdateSolrServer client from SolrJ to send events in batches as
> SolrInputDocuments using multipe worker threads to Apache Solr.
>
> The number of documents/events per batch and the number of worker threads
> can be configured.
>
> There is a default serializer as well.
>
> If you have the chance please try it out and let me know what you think.
>
> It would be great to gather some feedback from users before finalizing the
> features and configuration options for this sink.
>
> Instructions are available here on how to set up and configure this sink.
>
>
> https://cwiki.apache.org/confluence/display/FLUME/How+to+Setup+Solr+Sink+for+Flume
>
> JAR files for the feature as well as its dependencies are available here
>
> https://issues.apache.org/jira/browse/FLUME-1687
>