You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Eric DEWITTE <er...@iptwins.com> on 2020/05/18 12:30:13 UTC

problem to migrate solr 4.6 to 8.4

Hello,

I need help because I'm stuck on a data import.

I have an old server under solr 4.6 and I wanted to import data from it to a new server 8.4

The old server has a pretty big core with 352 million records...

I managed to import in increments of 100,000 up to 169 million, but since then I've been getting these error messages:
Exception in thread "main" org.apache.solr.client.solrj.SolrServerException: Timeout occurred while waiting response from server at: http://127.0.0.1:8984/solr/core1
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:676)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:265)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:1003)
        at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:1018)
        at Migrator.main(Migrator.java:43)
Caused by: java.net.SocketTimeoutException: Read timed out

I tried to lower the slice (50,000 instead of 100,000) but same effects.

I don't know where I can go to increase the timeout or if I have to do something else to be able to continue and finish my import.

Thank you

E.D.

Re: problem to migrate solr 4.6 to 8.4

Posted by Erick Erickson <er...@gmail.com>.
First, did you check the Solr log on the server that’s getting the
docs to index? Sometimes there’s a better error message/stack trace
there.

Second, although people do use DIH for production, I tend to think
of it as a PoC tool. You can pretty easily write a SolrJ program
that does the import, and that gives you a lot more control. Here;s
a sample of doing that:

https://lucidworks.com/post/indexing-with-solrj/

Best,
Erick

> On May 18, 2020, at 8:30 AM, Eric DEWITTE <er...@iptwins.com> wrote:
> 
> Hello,
> 
> I need help because I'm stuck on a data import.
> 
> I have an old server under solr 4.6 and I wanted to import data from it to a new server 8.4
> 
> The old server has a pretty big core with 352 million records...
> 
> I managed to import in increments of 100,000 up to 169 million, but since then I've been getting these error messages:
> Exception in thread "main" org.apache.solr.client.solrj.SolrServerException: Timeout occurred while waiting response from server at: http://127.0.0.1:8984/solr/core1
>        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:676)
>        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:265)
>        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
>        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
>        at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:1003)
>        at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:1018)
>        at Migrator.main(Migrator.java:43)
> Caused by: java.net.SocketTimeoutException: Read timed out
> 
> I tried to lower the slice (50,000 instead of 100,000) but same effects.
> 
> I don't know where I can go to increase the timeout or if I have to do something else to be able to continue and finish my import.
> 
> Thank you
> 
> E.D.