[jira] [Commented] (FLUME-1687) Solr Sink

["Israel Ekpo (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLUME-1687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13614528#comment-13614528 ] 

Israel Ekpo commented on FLUME-1687:
------------------------------------

I think this is a cool idea.

This could be a great alternative to the ElasticSearchSink.

There are some folks that have experience with Apache Solr but do not necessarily understand how to get ElasticSearch up and running.

Having a SolrSink as an alternative could be very helpful in creating a user interface for searching through event and log data collected with Flume using Apache Solr.

In ElasticSearch, the data sent to the Sink can be partitioned using the date (yyyy-MM-dd). With the SolrSink, the partitioning of the captured data by date can be done in a manner similar to ElasticSearch via the CREATE INDEX feature of CoreAdmin

http://wiki.apache.org/solr/CoreAdmin#CREATE

The only downside is that unlike ElasticSearch, where no pre-existing schemas are required, with Apache Solr, the new core can only be created based on a pre-existing instanceDir, solrconfig.xml, and schema.xml files.


                
> Solr Sink
> ---------
>
>                 Key: FLUME-1687
>                 URL: https://issues.apache.org/jira/browse/FLUME-1687
>             Project: Flume
>          Issue Type: Wish
>          Components: Sinks+Sources
>    Affects Versions: v1.2.0
>            Reporter: wolfgang hoschek
>
> Some use cases need near real time full text indexing of data through Flume into Solr, where a Flume sink can write directly to a Solr search server. This is a scalable way to provide low latency querying and data acquisition. It complements (rather than replaces) use cases based on Map Reduce batch analysis of HDFS data.
> Solr has a client API that uses REST to add documents to a Solr server, which in turn is based on Lucene. A Solr Sink can extract documents from flume events and forward them to Solr.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira