You are viewing a plain text version of this content. The canonical link for it is here.

Posted to solr-user@lucene.apache.org by Raymond Xie <xi...@gmail.com> on 2018/05/14 03:39:00 UTC

Multi threading indexing

Hello,

I have a huge amount of data (TB level) to be indexed, I am wondering if
anyone can share your idea/code to do the multithreading indexing?

*------------------------------------------------*
*Sincerely yours,*


*Raymond*

Re: Multi threading indexing

Posted by Rahul Singh <ra...@gmail.com>.

Can try to leverage Spark to index. Or Kafka Connect with SolR.

--
Rahul Singh
rahul.singh@anant.us

Anant Corporation

On May 14, 2018, 2:03 AM -0500, Mikhail Khludnev <mk...@apache.org>, wrote:
> A few years ago I provided server side concurrency "booster"
> https://issues.apache.org/jira/browse/SOLR-3585.
> But now, I'd rather suppose it's client-side (or ETL) duty.
>
> On Mon, May 14, 2018 at 6:39 AM, Raymond Xie <xi...@gmail.com> wrote:
>
> > Hello,
> >
> > I have a huge amount of data (TB level) to be indexed, I am wondering if
> > anyone can share your idea/code to do the multithreading indexing?
> >
> > *------------------------------------------------*
> > *Sincerely yours,*
> >
> >
> > *Raymond*
> >
>
>
>
> --
> Sincerely yours
> Mikhail Khludnev

Re: Multi threading indexing

Posted by Mikhail Khludnev <mk...@apache.org>.

A few years ago I provided server side concurrency "booster"
https://issues.apache.org/jira/browse/SOLR-3585.
But now, I'd rather suppose it's client-side (or ETL) duty.

On Mon, May 14, 2018 at 6:39 AM, Raymond Xie <xi...@gmail.com> wrote:

> Hello,
>
> I have a huge amount of data (TB level) to be indexed, I am wondering if
> anyone can share your idea/code to do the multithreading indexing?
>
> *------------------------------------------------*
> *Sincerely yours,*
>
>
> *Raymond*
>

-- 
Sincerely yours
Mikhail Khludnev