You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by "Rahul Bhawsar (JIRA)" <ji...@apache.org> on 2013/09/14 21:33:52 UTC
[jira] [Commented] (DERBY-6309) Embedded Derby throws
java.lang.OutOfMemoryError: Java heap space while importing data
[ https://issues.apache.org/jira/browse/DERBY-6309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767549#comment-13767549 ]
Rahul Bhawsar commented on DERBY-6309:
--------------------------------------
Hi Manikandan,
Could you use the property "derby.storage.sortBufferMax" to reduce the sortBuffer size to 512 bytes.
I am new developer; however, by looking at the code it looks like to me that it should reduce the memory consumption of sortBuffer, and should save you from OutOfMemoryError.
Thanks,
Rahul
> Embedded Derby throws java.lang.OutOfMemoryError: Java heap space while importing data
> --------------------------------------------------------------------------------------
>
> Key: DERBY-6309
> URL: https://issues.apache.org/jira/browse/DERBY-6309
> Project: Derby
> Issue Type: Bug
> Affects Versions: 10.10.1.1
> Reporter: Manikandan Chellappan
> Attachments: DerbyLeakSuspects.jpg
>
>
> While importing large files (> 500 MB) using SYSCS_UTIL.SYSCS_IMPORT_DATA procedure, derby runs out of heap space. The java process which is running the derby has 2GB of heap space. The following exception is logged in the derby.log.
> ERROR 38000: The exception 'java.lang.OutOfMemoryError: Java heap space' was thrown while evaluating an expression.
> at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
> at org.apache.derby.iapi.error.StandardException.unexpectedUserException(Unknown Source)
> at org.apache.derby.impl.sql.execute.VTIResultSet.getNextRowCore(Unknown Source)
> at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(Unknown Source)
> at org.apache.derby.impl.sql.execute.NormalizeResultSet.getNextRowCore(Unknown Source)
> at org.apache.derby.impl.sql.execute.NoPutResultSetImpl.getNextRowFromRowSource(Unknown Source)
> at org.apache.derby.impl.store.access.heap.HeapController.load(Unknown Source)
> at org.apache.derby.impl.store.access.heap.Heap.load(Unknown Source)
> at org.apache.derby.impl.store.access.RAMTransaction.loadConglomerate(Unknown Source)
> at org.apache.derby.impl.store.access.RAMTransaction.recreateAndLoadConglomerate(Unknown Source)
> at org.apache.derby.impl.sql.execute.InsertResultSet.bulkInsertCore(Unknown Source)
> at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source)
> at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
> at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
> at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
> at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
> at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeLargeUpdate(Unknown Source)
> at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source)
> at org.apache.derby.impl.load.Import.performImport(Unknown Source)
> at org.apache.derby.impl.load.Import.importData(Unknown Source)
> at org.apache.derby.catalog.SystemProcedures.SYSCS_IMPORT_DATA(Unknown Source)
> at org.apache.derby.exe.ace3c78554x013fxcb2cx4da6x00000f7e08680.g0(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor176.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.derby.impl.services.reflect.ReflectMethod.invoke(Unknown Source)
> at org.apache.derby.impl.sql.execute.CallStatementResultSet.open(Unknown Source)
> at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
> at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
> at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
> at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
> at org.apache.derby.impl.jdbc.EmbedPreparedStatement.execute(Unknown Source)
> at oracle.retail.stores.iddi.IDDIDerbyWriter.importTable(IDDIDerbyWriter.java:233)
> at oracle.retail.stores.domain.iddi.OfflineDBProducer$ImportThread.call(OfflineDBProducer.java:488)
> at oracle.retail.stores.domain.iddi.OfflineDBProducer$ImportThread.call(OfflineDBProducer.java:1)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> Caused by: java.lang.OutOfMemoryError: Java heap space
> at org.apache.derby.impl.load.ImportReadData.readNextDelimitedRow(Unknown Source)
> at org.apache.derby.impl.load.ImportReadData.readNextRow(Unknown Source)
> at org.apache.derby.impl.load.ImportAbstract.next(Unknown Source)
> at org.apache.derby.impl.sql.execute.VTIResultSet.getNextRowCore(Unknown Source)
> at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(Unknown Source)
> at org.apache.derby.impl.sql.execute.NormalizeResultSet.getNextRowCore(Unknown Source)
> at org.apache.derby.impl.sql.execute.NoPutResultSetImpl.getNextRowFromRowSource(Unknown Source)
> at org.apache.derby.impl.store.access.heap.HeapController.load(Unknown Source)
> at org.apache.derby.impl.store.access.heap.Heap.load(Unknown Source)
> at org.apache.derby.impl.store.access.RAMTransaction.loadConglomerate(Unknown Source)
> at org.apache.derby.impl.store.access.RAMTransaction.recreateAndLoadConglomerate(Unknown Source)
> at org.apache.derby.impl.sql.execute.InsertResultSet.bulkInsertCore(Unknown Source)
> at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source)
> at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
> at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
> at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
> at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
> at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeLargeUpdate(Unknown Source)
> at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source)
> at org.apache.derby.impl.load.Import.performImport(Unknown Source)
> at org.apache.derby.impl.load.Import.importData(Unknown Source)
> at org.apache.derby.catalog.SystemProcedures.SYSCS_IMPORT_DATA(Unknown Source)
> at org.apache.derby.exe.ace3c78554x013fxcb2cx4da6x00000f7e08680.g0(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor176.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.derby.impl.services.reflect.ReflectMethod.invoke(Unknown Source)
> at org.apache.derby.impl.sql.execute.CallStatementResultSet.open(Unknown Source)
> at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
> at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
> at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
> at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira