You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Sebastian Krebs (JIRA)" <ji...@apache.org> on 2015/06/16 16:22:01 UTC
[jira] [Updated] (SOLR-7691) SolrEntityProcessor as SubEntity
doesn't work with delta-import
[ https://issues.apache.org/jira/browse/SOLR-7691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sebastian Krebs updated SOLR-7691:
----------------------------------
Description:
I've used the {{SolrEntityProcessor}} as sub-entity in the dataimporter like this
{code:lang=xml}
<dataConfig>
<document name="products">
<entity
name="outer"
dataSource="my_datasource"
pk="id"
query="..."
deltaQuery="..."
deltaImportQuery="..."
>
<entity
name="solr"
processor="SolrEntityProcessor"
url="http://127.0.0.1:8983/solr/${solr.core.name}"
query="Xid:${outer.Xid}"
rows="1"
fl="Id,FieldA,FieldB"
wt="javabin"
/>
</entity>
</document>
</dataConfig>
{code}
Recently I decided to upgrade to 5.x, but the delta-import stopped working. At all it looks like the http-connection used by the {{SolrEntityProcessor}} is closed right _after_ the request/response, because the first document is indexed properly and for the second connection the dataimport fetches the record from the database, but after that exists
This is the stacktrace taken from the log
{code:lang=plain}
java.lang.RuntimeException: java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException: java.lang.IllegalStateException: Connection pool shut down
at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:270)
at org.apache.solr.handler.dataimport.DataImporter.doDeltaImport(DataImporter.java:444)
at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:482)
at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:461)
Caused by: java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException: java.lang.IllegalStateException: Connection pool shut down
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:416)
at org.apache.solr.handler.dataimport.DocBuilder.doDelta(DocBuilder.java:363)
at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:224)
... 3 more
Caused by: org.apache.solr.handler.dataimport.DataImportHandlerException: java.lang.IllegalStateException: Connection pool shut down
at org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:62)
at org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:246)
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:475)
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:514)
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:414)
... 5 more
Caused by: java.lang.IllegalStateException: Connection pool shut down
at org.apache.http.util.Asserts.check(Asserts.java:34)
at org.apache.http.pool.AbstractConnPool.lease(AbstractConnPool.java:184)
at org.apache.http.pool.AbstractConnPool.lease(AbstractConnPool.java:217)
at org.apache.http.impl.conn.PoolingClientConnectionManager.requestConnection(PoolingClientConnectionManager.java:184)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:415)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:466)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:958)
at org.apache.solr.handler.dataimport.SolrEntityProcessor.doQuery(SolrEntityProcessor.java:198)
at org.apache.solr.handler.dataimport.SolrEntityProcessor.buildIterator(SolrEntityProcessor.java:147)
at org.apache.solr.handler.dataimport.SolrEntityProcessor.nextRow(SolrEntityProcessor.java:128)
at org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:243)
... 8 more
{code}
was:
I've used the {{SolrEntityProcessor}} as sub-entity in the dataimporter like this
{code}
<dataConfig>
<document name="products">
<entity
name="outer"
dataSource="my_datasource"
pk="id"
query="..."
deltaQuery="..."
deltaImportQuery="..."
>
<entity
name="solr"
processor="SolrEntityProcessor"
url="http://127.0.0.1:8983/solr/${solr.core.name}"
query="Xid:${outer.Xid}"
rows="1"
fl="Id,FieldA,FieldB"
wt="javabin"
/>
</entity>
</document>
</dataConfig>
{code}
Recently I decided to upgrade to 5.x, but the delta-import stopped working. At all it looks like the http-connection used by the {{SolrEntityProcessor}} is closed right _after_ the request/response, because the first document is indexed properly and for the second connection the dataimport fetches the record from the database, but after that exists.
This is the stacktrace taken from the log
{code}
java.lang.RuntimeException: java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException: java.lang.IllegalStateException: Connection pool shut down
at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:270)
at org.apache.solr.handler.dataimport.DataImporter.doDeltaImport(DataImporter.java:444)
at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:482)
at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:461)
Caused by: java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException: java.lang.IllegalStateException: Connection pool shut down
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:416)
at org.apache.solr.handler.dataimport.DocBuilder.doDelta(DocBuilder.java:363)
at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:224)
... 3 more
Caused by: org.apache.solr.handler.dataimport.DataImportHandlerException: java.lang.IllegalStateException: Connection pool shut down
at org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:62)
at org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:246)
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:475)
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:514)
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:414)
... 5 more
Caused by: java.lang.IllegalStateException: Connection pool shut down
at org.apache.http.util.Asserts.check(Asserts.java:34)
at org.apache.http.pool.AbstractConnPool.lease(AbstractConnPool.java:184)
at org.apache.http.pool.AbstractConnPool.lease(AbstractConnPool.java:217)
at org.apache.http.impl.conn.PoolingClientConnectionManager.requestConnection(PoolingClientConnectionManager.java:184)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:415)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:466)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:958)
at org.apache.solr.handler.dataimport.SolrEntityProcessor.doQuery(SolrEntityProcessor.java:198)
at org.apache.solr.handler.dataimport.SolrEntityProcessor.buildIterator(SolrEntityProcessor.java:147)
at org.apache.solr.handler.dataimport.SolrEntityProcessor.nextRow(SolrEntityProcessor.java:128)
at org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:243)
... 8 more
{code}
> SolrEntityProcessor as SubEntity doesn't work with delta-import
> ---------------------------------------------------------------
>
> Key: SOLR-7691
> URL: https://issues.apache.org/jira/browse/SOLR-7691
> Project: Solr
> Issue Type: Bug
> Components: contrib - DataImportHandler
> Affects Versions: 5.0, 5.1, 5.2, 5.2.1
> Reporter: Sebastian Krebs
> Priority: Minor
>
> I've used the {{SolrEntityProcessor}} as sub-entity in the dataimporter like this
> {code:lang=xml}
> <dataConfig>
> <document name="products">
> <entity
> name="outer"
> dataSource="my_datasource"
> pk="id"
> query="..."
> deltaQuery="..."
> deltaImportQuery="..."
> >
> <entity
> name="solr"
> processor="SolrEntityProcessor"
> url="http://127.0.0.1:8983/solr/${solr.core.name}"
> query="Xid:${outer.Xid}"
> rows="1"
> fl="Id,FieldA,FieldB"
> wt="javabin"
> />
> </entity>
> </document>
> </dataConfig>
> {code}
> Recently I decided to upgrade to 5.x, but the delta-import stopped working. At all it looks like the http-connection used by the {{SolrEntityProcessor}} is closed right _after_ the request/response, because the first document is indexed properly and for the second connection the dataimport fetches the record from the database, but after that exists
> This is the stacktrace taken from the log
> {code:lang=plain}
> java.lang.RuntimeException: java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException: java.lang.IllegalStateException: Connection pool shut down
> at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:270)
> at org.apache.solr.handler.dataimport.DataImporter.doDeltaImport(DataImporter.java:444)
> at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:482)
> at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:461)
> Caused by: java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException: java.lang.IllegalStateException: Connection pool shut down
> at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:416)
> at org.apache.solr.handler.dataimport.DocBuilder.doDelta(DocBuilder.java:363)
> at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:224)
> ... 3 more
> Caused by: org.apache.solr.handler.dataimport.DataImportHandlerException: java.lang.IllegalStateException: Connection pool shut down
> at org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:62)
> at org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:246)
> at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:475)
> at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:514)
> at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:414)
> ... 5 more
> Caused by: java.lang.IllegalStateException: Connection pool shut down
> at org.apache.http.util.Asserts.check(Asserts.java:34)
> at org.apache.http.pool.AbstractConnPool.lease(AbstractConnPool.java:184)
> at org.apache.http.pool.AbstractConnPool.lease(AbstractConnPool.java:217)
> at org.apache.http.impl.conn.PoolingClientConnectionManager.requestConnection(PoolingClientConnectionManager.java:184)
> at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:415)
> at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:466)
> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235)
> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227)
> at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
> at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943)
> at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:958)
> at org.apache.solr.handler.dataimport.SolrEntityProcessor.doQuery(SolrEntityProcessor.java:198)
> at org.apache.solr.handler.dataimport.SolrEntityProcessor.buildIterator(SolrEntityProcessor.java:147)
> at org.apache.solr.handler.dataimport.SolrEntityProcessor.nextRow(SolrEntityProcessor.java:128)
> at org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:243)
> ... 8 more
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org