You are viewing a plain text version of this content. The canonical link for it is here.

Posted to solr-user@lucene.apache.org by Mark <st...@gmail.com> on 2011/12/02 01:57:37 UTC

Multithreaded DIH bug

I'm trying to use multiple threads with DIH but I keep receiving the 
following error.. "Operation not allowed after ResultSet closed"

Is there anyway I can fix this?

Dec 1, 2011 4:38:47 PM org.apache.solr.common.SolrException log
SEVERE: Full Import failed:java.lang.RuntimeException: Error in 
multi-threaded import
     at 
org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:268)
     at 
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:187)
     at 
org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:359)
     at 
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:427)
     at 
org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:408)
Caused by: 
org.apache.solr.handler.dataimport.DataImportHandlerException: 
java.sql.SQLException: Operation not allowed after ResultSet closed
     at 
org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:64)
     at 
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.hasnext(JdbcDataSource.java:339)
     at 
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.access$600(JdbcDataSource.java:228)
     at 
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator$1.hasNext(JdbcDataSource.java:262)
     at 
org.apache.solr.handler.dataimport.CachedSqlEntityProcessor.getAllNonCachedRows(CachedSqlEntityProcessor.java:72)
     at 
org.apache.solr.handler.dataimport.EntityProcessorBase.getIdCacheData(EntityProcessorBase.java:201)
     at 
org.apache.solr.handler.dataimport.CachedSqlEntityProcessor.nextRow(CachedSqlEntityProcessor.java:60)
     at 
org.apache.solr.handler.dataimport.ThreadedEntityProcessorWrapper.nextRow(ThreadedEntityProcessorWrapper.java:84)
     at 
org.apache.solr.handler.dataimport.DocBuilder$EntityRunner.runAThread(DocBuilder.java:449)
     at 
org.apache.solr.handler.dataimport.DocBuilder$EntityRunner.run(DocBuilder.java:402)
     at 
org.apache.solr.handler.dataimport.DocBuilder$EntityRunner.runAThread(DocBuilder.java:469)
     at 
org.apache.solr.handler.dataimport.DocBuilder$EntityRunner.access$000(DocBuilder.java:356)
     at 
org.apache.solr.handler.dataimport.DocBuilder$EntityRunner$1.run(DocBuilder.java:409)
     at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
     at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
     at java.lang.Thread.run(Thread.java:636)
Caused by: java.sql.SQLException: Operation not allowed after ResultSet 
closed
     at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1073)
     at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:987)
     at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:982)
     at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:927)
     at com.mysql.jdbc.ResultSetImpl.checkClosed(ResultSetImpl.java:794)
     at com.mysql.jdbc.ResultSetImpl.next(ResultSetImpl.java:7139)
     at 
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.hasnext(JdbcDataSource.java:331)
     ... 14 more

Re: Multithreaded DIH bug

Posted by Mark <st...@gmail.com>.

Thanks for the info

On 12/2/11 1:29 AM, Mikhail Khludnev wrote:
> Hello,
>
> AFAIK Particularly this exception is not a big deal. It's just one of the
> evidence of the fact that CachedSqlEntityProcessor doesn't work in multiple
> threads at 3.x and 4.0. It's discussed at
> http://search-lucene.com/m/0DNn32L2UBv
>
> the most problem here is the following messages in the log
>
> org.apache.solr.handler.dataimport.*ThreadedEntityProcessorWrapper. nextRow*
> *arow : null*
>
> Some time ago I did the patch for 3.4 (pretty raw) you can try it
> http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201110.mbox/browser
> I plan (but only plan, sorry) to address it at 4.0 where SOLR-2382
> refactoring has been applied recently.
>
> Regards
>
> On Fri, Dec 2, 2011 at 4:57 AM, Mark<st...@gmail.com>  wrote:
>
>> I'm trying to use multiple threads with DIH but I keep receiving the
>> following error.. "Operation not allowed after ResultSet closed"
>>
>> Is there anyway I can fix this?
>>
>> Dec 1, 2011 4:38:47 PM org.apache.solr.common.**SolrException log
>> SEVERE: Full Import failed:java.lang.**RuntimeException: Error in
>> multi-threaded import
>>     at org.apache.solr.handler.**dataimport.DocBuilder.**
>> doFullDump(DocBuilder.java:**268)
>>     at org.apache.solr.handler.**dataimport.DocBuilder.execute(**
>> DocBuilder.java:187)
>>     at org.apache.solr.handler.**dataimport.DataImporter.**
>> doFullImport(DataImporter.**java:359)
>>     at org.apache.solr.handler.**dataimport.DataImporter.**
>> runCmd(DataImporter.java:427)
>>     at org.apache.solr.handler.**dataimport.DataImporter$1.run(**
>> DataImporter.java:408)
>> Caused by: org.apache.solr.handler.**dataimport.**DataImportHandlerException:
>> java.sql.SQLException: Operation not allowed after ResultSet closed
>>     at org.apache.solr.handler.**dataimport.**DataImportHandlerException.**
>> wrapAndThrow(**DataImportHandlerException.**java:64)
>>     at org.apache.solr.handler.**dataimport.JdbcDataSource$**
>> ResultSetIterator.hasnext(**JdbcDataSource.java:339)
>>     at org.apache.solr.handler.**dataimport.JdbcDataSource$**
>> ResultSetIterator.access$600(**JdbcDataSource.java:228)
>>     at org.apache.solr.handler.**dataimport.JdbcDataSource$**
>> ResultSetIterator$1.hasNext(**JdbcDataSource.java:262)
>>     at org.apache.solr.handler.**dataimport.**CachedSqlEntityProcessor.**
>> getAllNonCachedRows(**CachedSqlEntityProcessor.java:**72)
>>     at org.apache.solr.handler.**dataimport.**EntityProcessorBase.**
>> getIdCacheData(**EntityProcessorBase.java:201)
>>     at org.apache.solr.handler.**dataimport.**CachedSqlEntityProcessor.**
>> nextRow(**CachedSqlEntityProcessor.java:**60)
>>     at org.apache.solr.handler.**dataimport.**
>> ThreadedEntityProcessorWrapper**.nextRow(**ThreadedEntityProcessorWrapper*
>> *.java:84)
>>     at org.apache.solr.handler.**dataimport.DocBuilder$**
>> EntityRunner.runAThread(**DocBuilder.java:449)
>>     at org.apache.solr.handler.**dataimport.DocBuilder$**
>> EntityRunner.run(DocBuilder.**java:402)
>>     at org.apache.solr.handler.**dataimport.DocBuilder$**
>> EntityRunner.runAThread(**DocBuilder.java:469)
>>     at org.apache.solr.handler.**dataimport.DocBuilder$**
>> EntityRunner.access$000(**DocBuilder.java:356)
>>     at org.apache.solr.handler.**dataimport.DocBuilder$**
>> EntityRunner$1.run(DocBuilder.**java:409)
>>     at java.util.concurrent.**ThreadPoolExecutor.runWorker(**
>> ThreadPoolExecutor.java:1110)
>>     at java.util.concurrent.**ThreadPoolExecutor$Worker.run(**
>> ThreadPoolExecutor.java:603)
>>     at java.lang.Thread.run(Thread.**java:636)
>> Caused by: java.sql.SQLException: Operation not allowed after ResultSet
>> closed
>>     at com.mysql.jdbc.SQLError.**createSQLException(SQLError.**java:1073)
>>     at com.mysql.jdbc.SQLError.**createSQLException(SQLError.**java:987)
>>     at com.mysql.jdbc.SQLError.**createSQLException(SQLError.**java:982)
>>     at com.mysql.jdbc.SQLError.**createSQLException(SQLError.**java:927)
>>     at com.mysql.jdbc.ResultSetImpl.**checkClosed(ResultSetImpl.**java:794)
>>     at com.mysql.jdbc.ResultSetImpl.**next(ResultSetImpl.java:7139)
>>     at org.apache.solr.handler.**dataimport.JdbcDataSource$**
>> ResultSetIterator.hasnext(**JdbcDataSource.java:331)
>>     ... 14 more
>>
>>
>

Re: Multithreaded DIH bug

Posted by Mikhail Khludnev <mk...@griddynamics.com>.

Hello,

AFAIK Particularly this exception is not a big deal. It's just one of the
evidence of the fact that CachedSqlEntityProcessor doesn't work in multiple
threads at 3.x and 4.0. It's discussed at
http://search-lucene.com/m/0DNn32L2UBv

the most problem here is the following messages in the log

org.apache.solr.handler.dataimport.*ThreadedEntityProcessorWrapper. nextRow*
*arow : null*

Some time ago I did the patch for 3.4 (pretty raw) you can try it
http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201110.mbox/browser
I plan (but only plan, sorry) to address it at 4.0 where SOLR-2382
refactoring has been applied recently.

Regards

On Fri, Dec 2, 2011 at 4:57 AM, Mark <st...@gmail.com> wrote:

> I'm trying to use multiple threads with DIH but I keep receiving the
> following error.. "Operation not allowed after ResultSet closed"
>
> Is there anyway I can fix this?
>
> Dec 1, 2011 4:38:47 PM org.apache.solr.common.**SolrException log
> SEVERE: Full Import failed:java.lang.**RuntimeException: Error in
> multi-threaded import
>    at org.apache.solr.handler.**dataimport.DocBuilder.**
> doFullDump(DocBuilder.java:**268)
>    at org.apache.solr.handler.**dataimport.DocBuilder.execute(**
> DocBuilder.java:187)
>    at org.apache.solr.handler.**dataimport.DataImporter.**
> doFullImport(DataImporter.**java:359)
>    at org.apache.solr.handler.**dataimport.DataImporter.**
> runCmd(DataImporter.java:427)
>    at org.apache.solr.handler.**dataimport.DataImporter$1.run(**
> DataImporter.java:408)
> Caused by: org.apache.solr.handler.**dataimport.**DataImportHandlerException:
> java.sql.SQLException: Operation not allowed after ResultSet closed
>    at org.apache.solr.handler.**dataimport.**DataImportHandlerException.**
> wrapAndThrow(**DataImportHandlerException.**java:64)
>    at org.apache.solr.handler.**dataimport.JdbcDataSource$**
> ResultSetIterator.hasnext(**JdbcDataSource.java:339)
>    at org.apache.solr.handler.**dataimport.JdbcDataSource$**
> ResultSetIterator.access$600(**JdbcDataSource.java:228)
>    at org.apache.solr.handler.**dataimport.JdbcDataSource$**
> ResultSetIterator$1.hasNext(**JdbcDataSource.java:262)
>    at org.apache.solr.handler.**dataimport.**CachedSqlEntityProcessor.**
> getAllNonCachedRows(**CachedSqlEntityProcessor.java:**72)
>    at org.apache.solr.handler.**dataimport.**EntityProcessorBase.**
> getIdCacheData(**EntityProcessorBase.java:201)
>    at org.apache.solr.handler.**dataimport.**CachedSqlEntityProcessor.**
> nextRow(**CachedSqlEntityProcessor.java:**60)
>    at org.apache.solr.handler.**dataimport.**
> ThreadedEntityProcessorWrapper**.nextRow(**ThreadedEntityProcessorWrapper*
> *.java:84)
>    at org.apache.solr.handler.**dataimport.DocBuilder$**
> EntityRunner.runAThread(**DocBuilder.java:449)
>    at org.apache.solr.handler.**dataimport.DocBuilder$**
> EntityRunner.run(DocBuilder.**java:402)
>    at org.apache.solr.handler.**dataimport.DocBuilder$**
> EntityRunner.runAThread(**DocBuilder.java:469)
>    at org.apache.solr.handler.**dataimport.DocBuilder$**
> EntityRunner.access$000(**DocBuilder.java:356)
>    at org.apache.solr.handler.**dataimport.DocBuilder$**
> EntityRunner$1.run(DocBuilder.**java:409)
>    at java.util.concurrent.**ThreadPoolExecutor.runWorker(**
> ThreadPoolExecutor.java:1110)
>    at java.util.concurrent.**ThreadPoolExecutor$Worker.run(**
> ThreadPoolExecutor.java:603)
>    at java.lang.Thread.run(Thread.**java:636)
> Caused by: java.sql.SQLException: Operation not allowed after ResultSet
> closed
>    at com.mysql.jdbc.SQLError.**createSQLException(SQLError.**java:1073)
>    at com.mysql.jdbc.SQLError.**createSQLException(SQLError.**java:987)
>    at com.mysql.jdbc.SQLError.**createSQLException(SQLError.**java:982)
>    at com.mysql.jdbc.SQLError.**createSQLException(SQLError.**java:927)
>    at com.mysql.jdbc.ResultSetImpl.**checkClosed(ResultSetImpl.**java:794)
>    at com.mysql.jdbc.ResultSetImpl.**next(ResultSetImpl.java:7139)
>    at org.apache.solr.handler.**dataimport.JdbcDataSource$**
> ResultSetIterator.hasnext(**JdbcDataSource.java:331)
>    ... 14 more
>
>


-- 
Sincerely yours
Mikhail Khludnev
Developer
Grid Dynamics
tel. 1-415-738-8644
Skype: mkhludnev
<http://www.griddynamics.com>
 <mk...@griddynamics.com>