You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Steve Sun <st...@anobii.com> on 2009/09/22 10:59:36 UTC
DIH import "out of memory" problem (batchSize and autoCommit not
working)
Hi,
I spent a whole day trying to make "batchSize" work for JdbcDataSource with
"org.postgresql.Driver", but got frustrated. At last I took a look into
DIH's source code and found that there's actually a bug in there. When JDBC
driver is placed in <solr-home>/lib (as instructed by DIHQuickStart page of
Solr wiki), but not in tomcat's lib directory, the JDBC connection will not
be configured as specified in the DIH configuration at all. Attributes like
autoCommit, readOnly and batchSize will be ignored. The fix is simple, have
attached my patch.
(contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/JdbcDataSource.java
r817524)
One work-around is: place your JDBC driver jar under tomcat's application
lib directory. e.g., tomcat/webapps/solr/WEB-INF/lib/
Have only tested with postgresql drivers, but seems the problem is generic
to all drivers placed in <solr-home>/lib.
Regards,
Steve
Re: DIH import "out of memory" problem (batchSize and autoCommit not
working)
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
On Tue, Sep 22, 2009 at 3:00 PM, Steve Sun <st...@anobii.com> wrote:
> Done.
> http://issues.apache.org/jira/browse/SOLR-1450
>
>
This is fixed in trunk now. Thanks Steve!
--
Regards,
Shalin Shekhar Mangar.
Re: DIH import "out of memory" problem (batchSize and autoCommit not
working)
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
On Tue, Sep 22, 2009 at 3:00 PM, Steve Sun <st...@anobii.com> wrote:
> Done.
> http://issues.apache.org/jira/browse/SOLR-1450
>
>
This is fixed in trunk now. Thanks Steve!
--
Regards,
Shalin Shekhar Mangar.
Re: DIH import "out of memory" problem (batchSize and autoCommit not
working)
Posted by Steve Sun <st...@anobii.com>.
2009/9/22 Shalin Shekhar Mangar <sh...@gmail.com>
> On Tue, Sep 22, 2009 at 2:29 PM, Steve Sun <st...@anobii.com> wrote:
>
> > Hi,
> >
> > I spent a whole day trying to make "batchSize" work for JdbcDataSource
> with
> > "org.postgresql.Driver", but got frustrated. At last I took a look into
> > DIH's source code and found that there's actually a bug in there. When
> JDBC
> > driver is placed in <solr-home>/lib (as instructed by DIHQuickStart page
> of
> > Solr wiki), but not in tomcat's lib directory, the JDBC connection will
> not
> > be configured as specified in the DIH configuration at all. Attributes
> like
> > autoCommit, readOnly and batchSize will be ignored. The fix is simple,
> have
> > attached my patch.
> >
> (contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/JdbcDataSource.java
> > r817524)
> >
> > One work-around is: place your JDBC driver jar under tomcat's application
> > lib directory. e.g., tomcat/webapps/solr/WEB-INF/lib/
> >
> > Have only tested with postgresql drivers, but seems the problem is
> generic
> > to all drivers placed in <solr-home>/lib.
> >
>
> Thanks Steve. The mailing list removed your attachment. Can you please open
> a jira issue and attach a patch there?
>
>
Done.
http://issues.apache.org/jira/browse/SOLR-1450
--
> Regards,
> Shalin Shekhar Mangar.
>
Re: DIH import "out of memory" problem (batchSize and autoCommit not
working)
Posted by Steve Sun <st...@anobii.com>.
2009/9/22 Shalin Shekhar Mangar <sh...@gmail.com>
> On Tue, Sep 22, 2009 at 2:29 PM, Steve Sun <st...@anobii.com> wrote:
>
> > Hi,
> >
> > I spent a whole day trying to make "batchSize" work for JdbcDataSource
> with
> > "org.postgresql.Driver", but got frustrated. At last I took a look into
> > DIH's source code and found that there's actually a bug in there. When
> JDBC
> > driver is placed in <solr-home>/lib (as instructed by DIHQuickStart page
> of
> > Solr wiki), but not in tomcat's lib directory, the JDBC connection will
> not
> > be configured as specified in the DIH configuration at all. Attributes
> like
> > autoCommit, readOnly and batchSize will be ignored. The fix is simple,
> have
> > attached my patch.
> >
> (contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/JdbcDataSource.java
> > r817524)
> >
> > One work-around is: place your JDBC driver jar under tomcat's application
> > lib directory. e.g., tomcat/webapps/solr/WEB-INF/lib/
> >
> > Have only tested with postgresql drivers, but seems the problem is
> generic
> > to all drivers placed in <solr-home>/lib.
> >
>
> Thanks Steve. The mailing list removed your attachment. Can you please open
> a jira issue and attach a patch there?
>
>
Done.
http://issues.apache.org/jira/browse/SOLR-1450
--
> Regards,
> Shalin Shekhar Mangar.
>
Re: DIH import "out of memory" problem (batchSize and autoCommit not
working)
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
On Tue, Sep 22, 2009 at 2:29 PM, Steve Sun <st...@anobii.com> wrote:
> Hi,
>
> I spent a whole day trying to make "batchSize" work for JdbcDataSource with
> "org.postgresql.Driver", but got frustrated. At last I took a look into
> DIH's source code and found that there's actually a bug in there. When JDBC
> driver is placed in <solr-home>/lib (as instructed by DIHQuickStart page of
> Solr wiki), but not in tomcat's lib directory, the JDBC connection will not
> be configured as specified in the DIH configuration at all. Attributes like
> autoCommit, readOnly and batchSize will be ignored. The fix is simple, have
> attached my patch.
> (contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/JdbcDataSource.java
> r817524)
>
> One work-around is: place your JDBC driver jar under tomcat's application
> lib directory. e.g., tomcat/webapps/solr/WEB-INF/lib/
>
> Have only tested with postgresql drivers, but seems the problem is generic
> to all drivers placed in <solr-home>/lib.
>
Thanks Steve. The mailing list removed your attachment. Can you please open
a jira issue and attach a patch there?
--
Regards,
Shalin Shekhar Mangar.
Re: DIH import "out of memory" problem (batchSize and autoCommit not
working)
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
On Tue, Sep 22, 2009 at 2:29 PM, Steve Sun <st...@anobii.com> wrote:
> Hi,
>
> I spent a whole day trying to make "batchSize" work for JdbcDataSource with
> "org.postgresql.Driver", but got frustrated. At last I took a look into
> DIH's source code and found that there's actually a bug in there. When JDBC
> driver is placed in <solr-home>/lib (as instructed by DIHQuickStart page of
> Solr wiki), but not in tomcat's lib directory, the JDBC connection will not
> be configured as specified in the DIH configuration at all. Attributes like
> autoCommit, readOnly and batchSize will be ignored. The fix is simple, have
> attached my patch.
> (contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/JdbcDataSource.java
> r817524)
>
> One work-around is: place your JDBC driver jar under tomcat's application
> lib directory. e.g., tomcat/webapps/solr/WEB-INF/lib/
>
> Have only tested with postgresql drivers, but seems the problem is generic
> to all drivers placed in <solr-home>/lib.
>
Thanks Steve. The mailing list removed your attachment. Can you please open
a jira issue and attach a patch there?
--
Regards,
Shalin Shekhar Mangar.