You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by Luc Caprini <Lu...@Sun.COM> on 2009/09/15 15:57:46 UTC

Problem with DataImportHandler and JDBC

Hi,

I'm trying using SOLR 1.4. and in my first test, I've got an issue with 
the DataImportHandler ...


Config : SOLR 1.4 on tomcat 5.5.27 with mysql driver 5.1.7 on mysql 5.1

My dataconfig is

<dataConfig>
<dataSource driver="com.mysql.jdbc.Driver" 
url="jdbc:mysql://localhost/testa" user="root" password="password"/>

<document>
<entity name="a" query="select * from a" >
<field column="b" name="named"/>
</entity>

</document>
</dataConfig>

And it generates this error

Unable to execute query: select * from a Processing Document # 1 at 
org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:72) 
at 
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:251) 
at 
org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:208) 
at 
org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:39) 
at 
org.apache.solr.handler.dataimport.DebugLogger$2.getData(DebugLogger.java:184) 
at 
org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:58) 
at 
org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:71) 
at 
org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:237) 
at 
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:339) 
at 
org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:225) 
at 
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:167) 
at 
org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:333) 
at 
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:393) 
at 
org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:203) 
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) 
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1301) at 
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338) 
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241) 
at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) 
at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) 
at 
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213) 
at 
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172) 
at 
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) 
at 
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117) 
at 
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108) 
at 
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174) 
at 
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:875) 
at 
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665) 
at 
org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528) 
at 
org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81) 
at 
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689) 
at java.lang.Thread.run(Unknown Source) Caused by: 
java.sql.SQLException: * Illegal value for setFetchSize()*. at 
com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1055) at 
com.mysql.jdbc.SQLError.createSQLException(SQLError.java:956) at 
com.mysql.jdbc.SQLError.createSQLException(SQLError.java:926) at 
com.mysql.jdbc.StatementImpl.setFetchSize(StatementImpl.java:2404) at 
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:240) 
... 30 more</str>

I tried to add fetchsize as a parameter :
<dataSource driver="com.mysql.jdbc.Driver" 
url="jdbc:mysql://localhost/testa" user="root" password="password" 
fetchSize="1"/>
but result is the same.

Thanks in advance for your assistance



Kind regards


Luc

-- 
<http://www.sun.com> 	* Luc Caprini *
Client Solutions Architect
*Sun Microsystems, Inc.*
13 Avenue Morane Saulnier
Velizy 78140 France
Phone +33 (0) 1 34 03 00 20
Mobile +33 (0) 6 12 30 16 22
Fax +33 (0) 1 34 03 10 11
Email Luc.Caprini@Sun.COM
	


Re: Problem with DataImportHandler and JDBC

Posted by Noble Paul നോബിള്‍ नोब्ळ् <no...@corp.aol.com>.
refer this http://wiki.apache.org/solr/DataImportHandlerFaq#head-149779b72761ab071c841879545256bdbbdc15d2

On Tue, Sep 15, 2009 at 11:40 PM, Luc Caprini <Lu...@sun.com> wrote:
> Hi
>
> Thanks for your quick answer. it is ok now...
>
> In fact, I try to read BLOB from my SQL Database.
>
> I'm trying the simplest test i can by only trying to read a blob column in a
> table.
>
> So my dataconfig is :
>
> <dataConfig>
> <dataSource driver="com.mysql.jdbc.Driver"
> url="jdbc:mysql://localhost/testa" user="root" password="password"
> batchSize="1"/>
> <document>
> <entity name="a"   transformer="ClobTransformer" query="select * from a">
> <field column="a"  name="mycol" clob="true" />
> </entity>
> </document>
> </dataConfig>
>
> Result is :
>
> <?xml version="1.0" encoding="UTF-8" ?>
> - <#> <response>
> - <#> <lst name="*responseHeader*">
> <int name="*status*">0</int>
> <int name="*QTime*">70</int>
> </lst>
> - <#> <lst name="*initArgs*">
> - <#> <lst name="*defaults*">
> <str name="*config*">data-conf.xml</str>
> </lst>
> </lst>
> <str name="*command*">full-import</str>
> <str name="*mode*">debug</str>
> <null name="*documents*" />
> - <#> <lst name="*verbose-output*">
> - <#> <lst name="*entity:a*">
> - <#> <lst name="*document#1*">
> <str name="*query*">select * from a</str>
> <str name="*time-taken*">0:0:0.40</str>
> <str>----------- row #1-------------</str>
> <str name="*a*">[B:[B@3f4ebd</str>
> <str>---------------------------------------------</str>
> - <#> <lst name="*transformer:ClobTransformer*">
> <str>---------------------------------------------</str>
> <str name="*a*">[B:[B@3f4ebd</str>
> <str>---------------------------------------------</str>
> </lst>
> </lst>
> <lst name="*document#1*" />
> </lst>
> </lst>
> <str name="*status*">idle</str>
> <str name="*importResponse*">Configuration Re-loaded sucessfully</str>
> - <#> <lst name="*statusMessages*">
> <str name="*Total Requests made to DataSource*">1</str>
> <str name="*Total Rows Fetched*">1</str>
> <str name="*Total Documents Skipped*">0</str>
> <str name="*Full Dump Started*">2009-09-15 20:06:12</str>
> <str name="*Time taken*">0:0:0.60</str>
> </lst>
> <str name="*WARNING*">This response format is experimental. It is likely to
> change in the future.</str>
> </response>
>
> As you can see, my column is not transformed.
>
> I read the code for ClobTransformer, and it seems that this class is only
> able to read Clob and not Blob.
>
> Am I wrong ?
>
> How i can achieve my goal ?
>
> Thanks in advance
>
> Regards
>
> Luc
> Le 15/09/2009 16:42, Noble Paul ???????  ?????? a écrit :
>>
>> the parameter name is batchSize
>>
>> by default it is set as 500
>>
>> On Tue, Sep 15, 2009 at 7:27 PM, Luc Caprini<Lu...@sun.com>  wrote:
>>
>>>
>>> Hi,
>>>
>>> I'm trying using SOLR 1.4. and in my first test, I've got an issue with
>>> the
>>> DataImportHandler ...
>>>
>>>
>>> Config : SOLR 1.4 on tomcat 5.5.27 with mysql driver 5.1.7 on mysql 5.1
>>>
>>> My dataconfig is
>>>
>>> <dataConfig>
>>> <dataSource driver="com.mysql.jdbc.Driver"
>>> url="jdbc:mysql://localhost/testa" user="root" password="password"/>
>>>
>>> <document>
>>> <entity name="a" query="select * from a">
>>> <field column="b" name="named"/>
>>> </entity>
>>>
>>> </document>
>>> </dataConfig>
>>>
>>> And it generates this error
>>>
>>> Unable to execute query: select * from a Processing Document # 1 at
>>>
>>> org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:72)
>>> at
>>>
>>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:251)
>>> at
>>>
>>> org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:208)
>>> at
>>>
>>> org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:39)
>>> at
>>>
>>> org.apache.solr.handler.dataimport.DebugLogger$2.getData(DebugLogger.java:184)
>>> at
>>>
>>> org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:58)
>>> at
>>>
>>> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:71)
>>> at
>>>
>>> org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:237)
>>> at
>>>
>>> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:339)
>>> at
>>>
>>> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:225)
>>> at
>>>
>>> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:167)
>>> at
>>>
>>> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:333)
>>> at
>>>
>>> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:393)
>>> at
>>>
>>> org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:203)
>>> at
>>>
>>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
>>> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1301) at
>>>
>>> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
>>> at
>>>
>>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
>>> at
>>>
>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
>>> at
>>>
>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
>>> at
>>>
>>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
>>> at
>>>
>>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172)
>>> at
>>>
>>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
>>> at
>>>
>>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
>>> at
>>>
>>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
>>> at
>>>
>>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
>>> at
>>>
>>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:875)
>>> at
>>>
>>> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
>>> at
>>>
>>> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
>>> at
>>>
>>> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
>>> at
>>>
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
>>> at java.lang.Thread.run(Unknown Source) Caused by: java.sql.SQLException:
>>> *
>>> Illegal value for setFetchSize()*. at
>>> com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1055) at
>>> com.mysql.jdbc.SQLError.createSQLException(SQLError.java:956) at
>>> com.mysql.jdbc.SQLError.createSQLException(SQLError.java:926) at
>>> com.mysql.jdbc.StatementImpl.setFetchSize(StatementImpl.java:2404) at
>>>
>>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:240)
>>> ... 30 more</str>
>>>
>>> I tried to add fetchsize as a parameter :
>>> <dataSource driver="com.mysql.jdbc.Driver"
>>> url="jdbc:mysql://localhost/testa" user="root" password="password"
>>> fetchSize="1"/>
>>> but result is the same.
>>>
>>> Thanks in advance for your assistance
>>>
>>>
>>>
>>> Kind regards
>>>
>>>
>>> Luc
>>>
>>> --
>>> <http://www.sun.com>      * Luc Caprini *
>>> Client Solutions Architect
>>> *Sun Microsystems, Inc.*
>>> 13 Avenue Morane Saulnier
>>> Velizy 78140 France
>>> Phone +33 (0) 1 34 03 00 20
>>> Mobile +33 (0) 6 12 30 16 22
>>> Fax +33 (0) 1 34 03 10 11
>>> Email Luc.Caprini@Sun.COM
>>>
>>>
>>>
>>>
>>
>>
>>
>
>
> --
> <http://www.sun.com>    * Luc Caprini *
> Client Solutions Architect
> *Sun Microsystems, Inc.*
> 13 Avenue Morane Saulnier
> Velizy 78140 France
> Phone +33 (0) 1 34 03 00 20
> Mobile +33 (0) 6 12 30 16 22
> Fax +33 (0) 1 34 03 10 11
> Email Luc.Caprini@Sun.COM
>
>
>



-- 
-----------------------------------------------------
Noble Paul | Principal Engineer| AOL | http://aol.com

Re: Problem with DataImportHandler and JDBC

Posted by Luc Caprini <Lu...@Sun.COM>.
Hi

Thanks for your quick answer. it is ok now...

In fact, I try to read BLOB from my SQL Database.

I'm trying the simplest test i can by only trying to read a blob column 
in a table.

So my dataconfig is :

<dataConfig>
<dataSource driver="com.mysql.jdbc.Driver" 
url="jdbc:mysql://localhost/testa" user="root" password="password" 
batchSize="1"/>
<document>
<entity name="a"   transformer="ClobTransformer" query="select * from a">
<field column="a"  name="mycol" clob="true" />
</entity>
</document>
</dataConfig>

Result is :

<?xml version="1.0" encoding="UTF-8" ?>
- <#> <response>
- <#> <lst name="*responseHeader*">
<int name="*status*">0</int>
<int name="*QTime*">70</int>
</lst>
- <#> <lst name="*initArgs*">
- <#> <lst name="*defaults*">
<str name="*config*">data-conf.xml</str>
</lst>
</lst>
<str name="*command*">full-import</str>
<str name="*mode*">debug</str>
<null name="*documents*" />
- <#> <lst name="*verbose-output*">
- <#> <lst name="*entity:a*">
- <#> <lst name="*document#1*">
<str name="*query*">select * from a</str>
<str name="*time-taken*">0:0:0.40</str>
<str>----------- row #1-------------</str>
<str name="*a*">[B:[B@3f4ebd</str>
<str>---------------------------------------------</str>
- <#> <lst name="*transformer:ClobTransformer*">
<str>---------------------------------------------</str>
<str name="*a*">[B:[B@3f4ebd</str>
<str>---------------------------------------------</str>
</lst>
</lst>
<lst name="*document#1*" />
</lst>
</lst>
<str name="*status*">idle</str>
<str name="*importResponse*">Configuration Re-loaded sucessfully</str>
- <#> <lst name="*statusMessages*">
<str name="*Total Requests made to DataSource*">1</str>
<str name="*Total Rows Fetched*">1</str>
<str name="*Total Documents Skipped*">0</str>
<str name="*Full Dump Started*">2009-09-15 20:06:12</str>
<str name="*Time taken*">0:0:0.60</str>
</lst>
<str name="*WARNING*">This response format is experimental. It is likely 
to change in the future.</str>
</response>

As you can see, my column is not transformed.

I read the code for ClobTransformer, and it seems that this class is 
only able to read Clob and not Blob.

Am I wrong ?

How i can achieve my goal ?

Thanks in advance

Regards

Luc
Le 15/09/2009 16:42, Noble Paul ???????  ?????? a écrit :
> the parameter name is batchSize
>
> by default it is set as 500
>
> On Tue, Sep 15, 2009 at 7:27 PM, Luc Caprini<Lu...@sun.com>  wrote:
>    
>> Hi,
>>
>> I'm trying using SOLR 1.4. and in my first test, I've got an issue with the
>> DataImportHandler ...
>>
>>
>> Config : SOLR 1.4 on tomcat 5.5.27 with mysql driver 5.1.7 on mysql 5.1
>>
>> My dataconfig is
>>
>> <dataConfig>
>> <dataSource driver="com.mysql.jdbc.Driver"
>> url="jdbc:mysql://localhost/testa" user="root" password="password"/>
>>
>> <document>
>> <entity name="a" query="select * from a">
>> <field column="b" name="named"/>
>> </entity>
>>
>> </document>
>> </dataConfig>
>>
>> And it generates this error
>>
>> Unable to execute query: select * from a Processing Document # 1 at
>> org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:72)
>> at
>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:251)
>> at
>> org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:208)
>> at
>> org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:39)
>> at
>> org.apache.solr.handler.dataimport.DebugLogger$2.getData(DebugLogger.java:184)
>> at
>> org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:58)
>> at
>> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:71)
>> at
>> org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:237)
>> at
>> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:339)
>> at
>> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:225)
>> at
>> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:167)
>> at
>> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:333)
>> at
>> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:393)
>> at
>> org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:203)
>> at
>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
>> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1301) at
>> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
>> at
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
>> at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
>> at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
>> at
>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
>> at
>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172)
>> at
>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
>> at
>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
>> at
>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
>> at
>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
>> at
>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:875)
>> at
>> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
>> at
>> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
>> at
>> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
>> at
>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
>> at java.lang.Thread.run(Unknown Source) Caused by: java.sql.SQLException: *
>> Illegal value for setFetchSize()*. at
>> com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1055) at
>> com.mysql.jdbc.SQLError.createSQLException(SQLError.java:956) at
>> com.mysql.jdbc.SQLError.createSQLException(SQLError.java:926) at
>> com.mysql.jdbc.StatementImpl.setFetchSize(StatementImpl.java:2404) at
>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:240)
>> ... 30 more</str>
>>
>> I tried to add fetchsize as a parameter :
>> <dataSource driver="com.mysql.jdbc.Driver"
>> url="jdbc:mysql://localhost/testa" user="root" password="password"
>> fetchSize="1"/>
>> but result is the same.
>>
>> Thanks in advance for your assistance
>>
>>
>>
>> Kind regards
>>
>>
>> Luc
>>
>> --
>> <http://www.sun.com>      * Luc Caprini *
>> Client Solutions Architect
>> *Sun Microsystems, Inc.*
>> 13 Avenue Morane Saulnier
>> Velizy 78140 France
>> Phone +33 (0) 1 34 03 00 20
>> Mobile +33 (0) 6 12 30 16 22
>> Fax +33 (0) 1 34 03 10 11
>> Email Luc.Caprini@Sun.COM
>>
>>
>>
>>      
>
>
>    


-- 
<http://www.sun.com> 	* Luc Caprini *
Client Solutions Architect
*Sun Microsystems, Inc.*
13 Avenue Morane Saulnier
Velizy 78140 France
Phone +33 (0) 1 34 03 00 20
Mobile +33 (0) 6 12 30 16 22
Fax +33 (0) 1 34 03 10 11
Email Luc.Caprini@Sun.COM
	


Re: Problem with DataImportHandler and JDBC

Posted by Noble Paul നോബിള്‍ नोब्ळ् <no...@corp.aol.com>.
the parameter name is batchSize

by default it is set as 500

On Tue, Sep 15, 2009 at 7:27 PM, Luc Caprini <Lu...@sun.com> wrote:
> Hi,
>
> I'm trying using SOLR 1.4. and in my first test, I've got an issue with the
> DataImportHandler ...
>
>
> Config : SOLR 1.4 on tomcat 5.5.27 with mysql driver 5.1.7 on mysql 5.1
>
> My dataconfig is
>
> <dataConfig>
> <dataSource driver="com.mysql.jdbc.Driver"
> url="jdbc:mysql://localhost/testa" user="root" password="password"/>
>
> <document>
> <entity name="a" query="select * from a" >
> <field column="b" name="named"/>
> </entity>
>
> </document>
> </dataConfig>
>
> And it generates this error
>
> Unable to execute query: select * from a Processing Document # 1 at
> org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:72)
> at
> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:251)
> at
> org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:208)
> at
> org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:39)
> at
> org.apache.solr.handler.dataimport.DebugLogger$2.getData(DebugLogger.java:184)
> at
> org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:58)
> at
> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:71)
> at
> org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:237)
> at
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:339)
> at
> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:225)
> at
> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:167)
> at
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:333)
> at
> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:393)
> at
> org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:203)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1301) at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
> at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:875)
> at
> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
> at
> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
> at
> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
> at
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
> at java.lang.Thread.run(Unknown Source) Caused by: java.sql.SQLException: *
> Illegal value for setFetchSize()*. at
> com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1055) at
> com.mysql.jdbc.SQLError.createSQLException(SQLError.java:956) at
> com.mysql.jdbc.SQLError.createSQLException(SQLError.java:926) at
> com.mysql.jdbc.StatementImpl.setFetchSize(StatementImpl.java:2404) at
> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:240)
> ... 30 more</str>
>
> I tried to add fetchsize as a parameter :
> <dataSource driver="com.mysql.jdbc.Driver"
> url="jdbc:mysql://localhost/testa" user="root" password="password"
> fetchSize="1"/>
> but result is the same.
>
> Thanks in advance for your assistance
>
>
>
> Kind regards
>
>
> Luc
>
> --
> <http://www.sun.com>    * Luc Caprini *
> Client Solutions Architect
> *Sun Microsystems, Inc.*
> 13 Avenue Morane Saulnier
> Velizy 78140 France
> Phone +33 (0) 1 34 03 00 20
> Mobile +33 (0) 6 12 30 16 22
> Fax +33 (0) 1 34 03 10 11
> Email Luc.Caprini@Sun.COM
>
>
>



-- 
-----------------------------------------------------
Noble Paul | Principal Engineer| AOL | http://aol.com