You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by "Steve Sun (JIRA)" <ji...@apache.org> on 2009/09/22 11:21:16 UTC
[jira] Created: (SOLR-1450) DIH import "out of memory" problem
(batchSize and autoCommit not working)
DIH import "out of memory" problem (batchSize and autoCommit not working)
-------------------------------------------------------------------------
Key: SOLR-1450
URL: https://issues.apache.org/jira/browse/SOLR-1450
Project: Solr
Issue Type: Bug
Components: contrib - DataImportHandler
Affects Versions: 1.3, 1.2, 1.1.0, 1.4, 1.5
Environment: Generic
Reporter: Steve Sun
When JDBC driver is placed in <solr-home>/lib (as instructed by DIHQuickStart page of Solr wiki), but not in tomcat's lib directory, the JDBC connection will not be configured as specified in the DIH configuration. Attributes like autoCommit, readOnly and batchSize will be ignored.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1450) DIH import "out of memory" problem
(batchSize and autoCommit not working)
Posted by "Steve Sun (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Sun updated SOLR-1450:
----------------------------
Attachment: jdbc-817524.loadClass.patch
Index: contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/JdbcDataSource.java
===================================================================
--- contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/JdbcDataSource.java (revision 817524)
+++ contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/JdbcDataSource.java (working copy)
@@ -153,6 +153,15 @@
"the jndi name : '"+jndiName +"' is not a valid javax.sql.DataSource");
}
}
+ } catch (SQLException e) {
+ // DriverManager does not allow you to use a driver which is not loaded through
+ // the class loader of the class which is trying to make the connection.
+ // This is a workaround for cases where the user puts the driver jar in the
+ // solr.home/lib or solr.home/core/lib directories.
+ Driver d = (Driver) DocBuilder.loadClass(driver, context.getSolrCore()).newInstance();
+ c = d.connect(url, initProps);
+ }
+ if (c != null) {
if (Boolean.parseBoolean(initProps.getProperty("readOnly"))) {
c.setReadOnly(true);
// Add other sane defaults
@@ -181,13 +190,6 @@
} else {
c.setHoldability(ResultSet.HOLD_CURSORS_OVER_COMMIT);
}
- } catch (SQLException e) {
- // DriverManager does not allow you to use a driver which is not loaded through
- // the class loader of the class which is trying to make the connection.
- // This is a workaround for cases where the user puts the driver jar in the
- // solr.home/lib or solr.home/core/lib directories.
- Driver d = (Driver) DocBuilder.loadClass(driver, context.getSolrCore()).newInstance();
- c = d.connect(url, initProps);
}
LOG.info("Time taken for getConnection(): "
+ (System.currentTimeMillis() - start));
> DIH import "out of memory" problem (batchSize and autoCommit not working)
> -------------------------------------------------------------------------
>
> Key: SOLR-1450
> URL: https://issues.apache.org/jira/browse/SOLR-1450
> Project: Solr
> Issue Type: Bug
> Components: contrib - DataImportHandler
> Affects Versions: 1.1.0, 1.2, 1.3, 1.4, 1.5
> Environment: Generic
> Reporter: Steve Sun
> Attachments: jdbc-817524.loadClass.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> When JDBC driver is placed in <solr-home>/lib (as instructed by DIHQuickStart page of Solr wiki), but not in tomcat's lib directory, the JDBC connection will not be configured as specified in the DIH configuration. Attributes like autoCommit, readOnly and batchSize will be ignored.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1450) DIH - Jdbc connection properties such
as batchSize are not applied if the driver jar is placed in solr_home/lib
Posted by "Steve Sun (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12771248#action_12771248 ]
Steve Sun commented on SOLR-1450:
---------------------------------
If Solr supports these attributes, yes. Otherwise, no.
> DIH - Jdbc connection properties such as batchSize are not applied if the driver jar is placed in solr_home/lib
> ---------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-1450
> URL: https://issues.apache.org/jira/browse/SOLR-1450
> Project: Solr
> Issue Type: Bug
> Components: contrib - DataImportHandler
> Affects Versions: 1.1.0, 1.2, 1.3, 1.4, 1.5
> Environment: Generic
> Reporter: Steve Sun
> Assignee: Shalin Shekhar Mangar
> Fix For: 1.4
>
> Attachments: jdbc-817524.loadClass.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> When JDBC driver is placed in <solr-home>/lib (as instructed by DIHQuickStart page of Solr wiki), but not in tomcat's lib directory, the JDBC connection will not be configured as specified in the DIH configuration. Attributes like autoCommit, readOnly and batchSize will be ignored.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1450) DIH - Jdbc connection properties such
as batchSize are not applied if the driver jar is placed in solr_home/lib
Posted by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12758225#action_12758225 ]
Shalin Shekhar Mangar commented on SOLR-1450:
---------------------------------------------
Thanks for the report and fix Steve!
> DIH - Jdbc connection properties such as batchSize are not applied if the driver jar is placed in solr_home/lib
> ---------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-1450
> URL: https://issues.apache.org/jira/browse/SOLR-1450
> Project: Solr
> Issue Type: Bug
> Components: contrib - DataImportHandler
> Affects Versions: 1.1.0, 1.2, 1.3, 1.4, 1.5
> Environment: Generic
> Reporter: Steve Sun
> Assignee: Shalin Shekhar Mangar
> Fix For: 1.4
>
> Attachments: jdbc-817524.loadClass.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> When JDBC driver is placed in <solr-home>/lib (as instructed by DIHQuickStart page of Solr wiki), but not in tomcat's lib directory, the JDBC connection will not be configured as specified in the DIH configuration. Attributes like autoCommit, readOnly and batchSize will be ignored.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1450) DIH - Jdbc connection properties such
as batchSize are not applied if the driver jar is placed in solr_home/lib
Posted by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shalin Shekhar Mangar updated SOLR-1450:
----------------------------------------
Summary: DIH - Jdbc connection properties such as batchSize are not applied if the driver jar is placed in solr_home/lib (was: DIH import "out of memory" problem (batchSize and autoCommit not working))
> DIH - Jdbc connection properties such as batchSize are not applied if the driver jar is placed in solr_home/lib
> ---------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-1450
> URL: https://issues.apache.org/jira/browse/SOLR-1450
> Project: Solr
> Issue Type: Bug
> Components: contrib - DataImportHandler
> Affects Versions: 1.1.0, 1.2, 1.3, 1.4, 1.5
> Environment: Generic
> Reporter: Steve Sun
> Assignee: Shalin Shekhar Mangar
> Fix For: 1.4
>
> Attachments: jdbc-817524.loadClass.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> When JDBC driver is placed in <solr-home>/lib (as instructed by DIHQuickStart page of Solr wiki), but not in tomcat's lib directory, the JDBC connection will not be configured as specified in the DIH configuration. Attributes like autoCommit, readOnly and batchSize will be ignored.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1450) DIH import "out of memory" problem
(batchSize and autoCommit not working)
Posted by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shalin Shekhar Mangar updated SOLR-1450:
----------------------------------------
Fix Version/s: 1.4
Assignee: Shalin Shekhar Mangar
> DIH import "out of memory" problem (batchSize and autoCommit not working)
> -------------------------------------------------------------------------
>
> Key: SOLR-1450
> URL: https://issues.apache.org/jira/browse/SOLR-1450
> Project: Solr
> Issue Type: Bug
> Components: contrib - DataImportHandler
> Affects Versions: 1.1.0, 1.2, 1.3, 1.4, 1.5
> Environment: Generic
> Reporter: Steve Sun
> Assignee: Shalin Shekhar Mangar
> Fix For: 1.4
>
> Attachments: jdbc-817524.loadClass.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> When JDBC driver is placed in <solr-home>/lib (as instructed by DIHQuickStart page of Solr wiki), but not in tomcat's lib directory, the JDBC connection will not be configured as specified in the DIH configuration. Attributes like autoCommit, readOnly and batchSize will be ignored.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-1450) DIH - Jdbc connection properties such
as batchSize are not applied if the driver jar is placed in solr_home/lib
Posted by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shalin Shekhar Mangar resolved SOLR-1450.
-----------------------------------------
Resolution: Fixed
Committed revision 817634.
> DIH - Jdbc connection properties such as batchSize are not applied if the driver jar is placed in solr_home/lib
> ---------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-1450
> URL: https://issues.apache.org/jira/browse/SOLR-1450
> Project: Solr
> Issue Type: Bug
> Components: contrib - DataImportHandler
> Affects Versions: 1.1.0, 1.2, 1.3, 1.4, 1.5
> Environment: Generic
> Reporter: Steve Sun
> Assignee: Shalin Shekhar Mangar
> Fix For: 1.4
>
> Attachments: jdbc-817524.loadClass.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> When JDBC driver is placed in <solr-home>/lib (as instructed by DIHQuickStart page of Solr wiki), but not in tomcat's lib directory, the JDBC connection will not be configured as specified in the DIH configuration. Attributes like autoCommit, readOnly and batchSize will be ignored.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (SOLR-1450) DIH import "out of memory"
problem (batchSize and autoCommit not working)
Posted by "Steve Sun (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12758181#action_12758181 ]
Steve Sun edited comment on SOLR-1450 at 9/22/09 2:28 AM:
----------------------------------------------------------
Move the connection setup code out of the try...catch block.
was (Author: steve.sun):
Index: contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/JdbcDataSource.java
===================================================================
--- contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/JdbcDataSource.java (revision 817524)
+++ contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/JdbcDataSource.java (working copy)
@@ -153,6 +153,15 @@
"the jndi name : '"+jndiName +"' is not a valid javax.sql.DataSource");
}
}
+ } catch (SQLException e) {
+ // DriverManager does not allow you to use a driver which is not loaded through
+ // the class loader of the class which is trying to make the connection.
+ // This is a workaround for cases where the user puts the driver jar in the
+ // solr.home/lib or solr.home/core/lib directories.
+ Driver d = (Driver) DocBuilder.loadClass(driver, context.getSolrCore()).newInstance();
+ c = d.connect(url, initProps);
+ }
+ if (c != null) {
if (Boolean.parseBoolean(initProps.getProperty("readOnly"))) {
c.setReadOnly(true);
// Add other sane defaults
@@ -181,13 +190,6 @@
} else {
c.setHoldability(ResultSet.HOLD_CURSORS_OVER_COMMIT);
}
- } catch (SQLException e) {
- // DriverManager does not allow you to use a driver which is not loaded through
- // the class loader of the class which is trying to make the connection.
- // This is a workaround for cases where the user puts the driver jar in the
- // solr.home/lib or solr.home/core/lib directories.
- Driver d = (Driver) DocBuilder.loadClass(driver, context.getSolrCore()).newInstance();
- c = d.connect(url, initProps);
}
LOG.info("Time taken for getConnection(): "
+ (System.currentTimeMillis() - start));
> DIH import "out of memory" problem (batchSize and autoCommit not working)
> -------------------------------------------------------------------------
>
> Key: SOLR-1450
> URL: https://issues.apache.org/jira/browse/SOLR-1450
> Project: Solr
> Issue Type: Bug
> Components: contrib - DataImportHandler
> Affects Versions: 1.1.0, 1.2, 1.3, 1.4, 1.5
> Environment: Generic
> Reporter: Steve Sun
> Attachments: jdbc-817524.loadClass.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> When JDBC driver is placed in <solr-home>/lib (as instructed by DIHQuickStart page of Solr wiki), but not in tomcat's lib directory, the JDBC connection will not be configured as specified in the DIH configuration. Attributes like autoCommit, readOnly and batchSize will be ignored.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1450) DIH - Jdbc connection properties such
as batchSize are not applied if the driver jar is placed in solr_home/lib
Posted by "Brian Mansell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12771051#action_12771051 ]
Brian Mansell commented on SOLR-1450:
-------------------------------------
Nice work!
Does this apply to JDBC connection attributes like autoReconnect and connectTimeout?
thanks,
--bemansell
> DIH - Jdbc connection properties such as batchSize are not applied if the driver jar is placed in solr_home/lib
> ---------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-1450
> URL: https://issues.apache.org/jira/browse/SOLR-1450
> Project: Solr
> Issue Type: Bug
> Components: contrib - DataImportHandler
> Affects Versions: 1.1.0, 1.2, 1.3, 1.4, 1.5
> Environment: Generic
> Reporter: Steve Sun
> Assignee: Shalin Shekhar Mangar
> Fix For: 1.4
>
> Attachments: jdbc-817524.loadClass.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> When JDBC driver is placed in <solr-home>/lib (as instructed by DIHQuickStart page of Solr wiki), but not in tomcat's lib directory, the JDBC connection will not be configured as specified in the DIH configuration. Attributes like autoCommit, readOnly and batchSize will be ignored.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.