You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by swapnil dubey <sw...@gmail.com> on 2010/10/15 13:42:38 UTC
problem on running fullimport
Hi,
I am using the full import option with the data-config file as mentioned
below
<dataConfig>
<dataSource type="JdbcDataSource" driver="com.mysql.jdbc.Driver"
url="jdbc:mysql://xxxx/xxx" user="xxx" password="xx" />
<document >
<entity name="yyy" query="select studentName from test1">
<field column="studentName" name="studentName" />
</entity>
</document>
</dataConfig>
on running the full-import option I am getting the error mentioned below.I
had already included the dataimport.properties file in my conf file.help me
to get the issue resolved
<response>
-
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">334</int>
</lst>
-
<lst name="initArgs">
-
<lst name="defaults">
<str name="config">data-config.xml</str>
</lst>
</lst>
<str name="command">full-import</str>
<str name="mode">debug</str>
<null name="documents"/>
-
<lst name="verbose-output">
-
<lst name="entity:test1">
-
<lst name="document#1">
<str name="query">select studentName from test1</str>
-
<str name="EXCEPTION">
org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to
execute query: select studentName from test1 Processing Document # 1
at
org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:72)
at
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:253)
at
org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:210)
at
org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:39)
at
org.apache.solr.handler.dataimport.DebugLogger$2.getData(DebugLogger.java:184)
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:58)
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:71)
at
org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:237)
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:357)
at
org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:242)
at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:180)
at
org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:331)
at
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:389)
at
org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:203)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
at
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
at org.mortbay.jetty.Server.handle(Server.java:285)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)
at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:821)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:513)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
at
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226)
at
org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
Caused by: java.sql.SQLException: Illegal value for setFetchSize().
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1075)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:989)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:984)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:929)
at com.mysql.jdbc.StatementImpl.setFetchSize(StatementImpl.java:2496)
at
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:242)
... 33 more
</str>
<str name="time-taken">0:0:0.50</str>
</lst>
</lst>
</lst>
<str name="status">idle</str>
<str name="importResponse">Configuration Re-loaded sucessfully</str>
-
<lst name="statusMessages">
<str name="Time Elapsed">0:0:0.299</str>
<str name="Total Requests made to DataSource">1</str>
<str name="Total Rows Fetched">0</str>
<str name="Total Documents Processed">0</str>
<str name="Total Documents Skipped">0</str>
<str name="Full Dump Started">2010-10-15 16:42:21</str>
<str name="">Indexing failed. Rolled back all changes.</str>
<str name="Rolledback">2010-10-15 16:42:21</str>
</lst>
-
<str name="WARNING">
This response format is experimental. It is likely to change in the future.
</str>
</response>
--
Regards
Swapnil Dubey
RE: problem on running fullimport
Posted by "Dyer, James" <Ja...@ingrambook.com>.
On your <dataSource /> tag, specify "batchSize" with a value that your db/driver allows. The hardcoded default is 500. If you set it to -1 it converts it to Integer.MIN_VALUE. See http://wiki.apache.org/solr/DataImportHandler#Configuring_JdbcDataSource , which recommends using this -1 value in the case of errors.
When executing queries, JdbcDataSource calls Statement.setFetchSize(batchSize) . Based on the java.sql.Statement api documentation, you can set this to 0 to retain your db/driver's default. See http://docs.oracle.com/javase/6/docs/api/java/sql/Statement.html#setFetchSize%28int%29
James Dyer
E-Commerce Systems
Ingram Content Group
(615) 213-4311
-----Original Message-----
From: pla [mailto:patrick.archibald@gmail.com]
Sent: Thursday, May 24, 2012 9:10 AM
To: solr-user@lucene.apache.org
Subject: Re: problem on running fullimport
Thanks Alexey Serba. I encountered the *java.sql.SQLException: Illegal value
for setFetchSize()* error after upgrading one of my servers to MySQL version
5.5.22.
PLA
--
View this message in context: http://lucene.472066.n3.nabble.com/problem-on-running-fullimport-tp1707206p3985924.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: problem on running fullimport
Posted by pla <pa...@gmail.com>.
Thanks Alexey Serba. I encountered the *java.sql.SQLException: Illegal value
for setFetchSize()* error after upgrading one of my servers to MySQL version
5.5.22.
PLA
--
View this message in context: http://lucene.472066.n3.nabble.com/problem-on-running-fullimport-tp1707206p3985924.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: problem on running fullimport
Posted by Alexey Serba <as...@gmail.com>.
" Caused by: java.sql.SQLException: Illegal value for setFetchSize(). "
Try to add batchSize="-1" to your data source declaration
http://wiki.apache.org/solr/DataImportHandlerFaq#I.27m_using_DataImportHandler_with_a_MySQL_database._My_table_is_huge_and_DataImportHandler_is_going_out_of_memory._Why_does_DataImportHandler_bring_everything_to_memory.3F
On Fri, Oct 15, 2010 at 3:42 PM, swapnil dubey <sw...@gmail.com> wrote:
> Hi,
>
> I am using the full import option with the data-config file as mentioned
> below
>
> <dataConfig>
> <dataSource type="JdbcDataSource" driver="com.mysql.jdbc.Driver"
> url="jdbc:mysql://xxxx/xxx" user="xxx" password="xx" />
> <document >
> <entity name="yyy" query="select studentName from test1">
> <field column="studentName" name="studentName" />
> </entity>
> </document>
> </dataConfig>
>
>
> on running the full-import option I am getting the error mentioned below.I
> had already included the dataimport.properties file in my conf file.help me
> to get the issue resolved
>
> <response>
> -
> <lst name="responseHeader">
> <int name="status">0</int>
> <int name="QTime">334</int>
> </lst>
> -
> <lst name="initArgs">
> -
> <lst name="defaults">
> <str name="config">data-config.xml</str>
> </lst>
> </lst>
> <str name="command">full-import</str>
> <str name="mode">debug</str>
> <null name="documents"/>
> -
> <lst name="verbose-output">
> -
> <lst name="entity:test1">
> -
> <lst name="document#1">
> <str name="query">select studentName from test1</str>
> -
> <str name="EXCEPTION">
> org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to
> execute query: select studentName from test1 Processing Document # 1
> at
> org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:72)
> at
> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:253)
> at
> org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:210)
> at
> org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:39)
> at
> org.apache.solr.handler.dataimport.DebugLogger$2.getData(DebugLogger.java:184)
> at
> org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:58)
> at
> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:71)
> at
> org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:237)
> at
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:357)
> at
> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:242)
> at
> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:180)
> at
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:331)
> at
> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:389)
> at
> org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:203)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
> at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
> at
> org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
> at org.mortbay.jetty.Server.handle(Server.java:285)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:821)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:513)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
> at
> org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226)
> at
> org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
> Caused by: java.sql.SQLException: Illegal value for setFetchSize().
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1075)
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:989)
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:984)
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:929)
> at com.mysql.jdbc.StatementImpl.setFetchSize(StatementImpl.java:2496)
> at
> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:242)
> ... 33 more
> </str>
> <str name="time-taken">0:0:0.50</str>
> </lst>
> </lst>
> </lst>
> <str name="status">idle</str>
> <str name="importResponse">Configuration Re-loaded sucessfully</str>
> -
> <lst name="statusMessages">
> <str name="Time Elapsed">0:0:0.299</str>
> <str name="Total Requests made to DataSource">1</str>
> <str name="Total Rows Fetched">0</str>
> <str name="Total Documents Processed">0</str>
> <str name="Total Documents Skipped">0</str>
> <str name="Full Dump Started">2010-10-15 16:42:21</str>
> <str name="">Indexing failed. Rolled back all changes.</str>
> <str name="Rolledback">2010-10-15 16:42:21</str>
> </lst>
> -
> <str name="WARNING">
> This response format is experimental. It is likely to change in the future.
> </str>
> </response>
>
> --
> Regards
> Swapnil Dubey
>
Re: problem on running fullimport
Posted by Ken Stanley <do...@gmail.com>.
On Fri, Oct 15, 2010 at 7:42 AM, swapnil dubey <sw...@gmail.com>wrote:
> Hi,
>
> I am using the full import option with the data-config file as mentioned
> below
>
> <dataConfig>
> <dataSource type="JdbcDataSource" driver="com.mysql.jdbc.Driver"
> url="jdbc:mysql://xxxx/xxx" user="xxx" password="xx" />
> <document >
> <entity name="yyy" query="select studentName from test1">
> <field column="studentName" name="studentName" />
> </entity>
> </document>
> </dataConfig>
>
>
> on running the full-import option I am getting the error mentioned below.I
> had already included the dataimport.properties file in my conf file.help me
> to get the issue resolved
>
> <response>
> -
> <lst name="responseHeader">
> <int name="status">0</int>
> <int name="QTime">334</int>
> </lst>
> -
> <lst name="initArgs">
> -
> <lst name="defaults">
> <str name="config">data-config.xml</str>
> </lst>
> </lst>
> <str name="command">full-import</str>
> <str name="mode">debug</str>
> <null name="documents"/>
> -
> <lst name="verbose-output">
> -
> <lst name="entity:test1">
> -
> <lst name="document#1">
> <str name="query">select studentName from test1</str>
> -
> <str name="EXCEPTION">
> org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to
> execute query: select studentName from test1 Processing Document # 1
> ...
>
> --
> Regards
> Swapnil Dubey
>
Swapnil,
Everything looks fine, except that in your entity definition you forgot to
define which datasource you wish to use. So if you add the
'dataSource="JdbcDataSource"' that should get rid of your exception. As a
reminder, the DataImportHandler wiki (
http://wiki.apache.org/solr/DataImportHandler) on Apache's website is very
helpful with learning how to use the DIH properly. It has helped me with
having a printed copy beside me for easy and quick reference.
- Ken