You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by "Mike Matrigali (JIRA)" <ji...@apache.org> on 2007/11/20 00:30:43 UTC
[jira] Created: (DERBY-3216) do row level lock space reclamation in
btree of indiv rows.
do row level lock space reclamation in btree of indiv rows.
-----------------------------------------------------------
Key: DERBY-3216
URL: https://issues.apache.org/jira/browse/DERBY-3216
Project: Derby
Issue Type: Bug
Components: Store
Affects Versions: 10.3.1.4
Reporter: Mike Matrigali
Assignee: Mike Matrigali
Priority: Minor
If you can't get a table level lock for btree space recovery in
the post commit thread, maybe you should at least reclaim the
rows on the page while you are at it. Use the same algorithm
as exists in BTreeController.java. row level shrink is a different
issue and won't be resolved by this.
Note there have been reports of "memory" leaks associated with this issue. This is because
currently if the work can not be done then we just queue it and move on. But in a stress situation
one may never get the required table lock to shrink the tree and thus the queue just keeps growing.
Note in many of these cases the app doesn't care if the page merge happens as it is just going to
insert more rows after the merge.
Also there is no need for a table level lock for a 1 page index as no merge is actually necessary.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-3216) do row level lock space reclamation
in btree of indiv rows.
Posted by "James Alan Shepherd (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-3216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12563083#action_12563083 ]
James Alan Shepherd commented on DERBY-3216:
--------------------------------------------
Possibly I have a resulting NPE in Derby 10.3.2.1 (10.3.1.4 does not show this behaviour)
FATAL 38406 [Main] (Start.java:153) - Start FAILED
org.springframework.transaction.TransactionSystemException: Could not commit JDBC transaction; nested exception is java.sql.SQLException: Java exception: ': java.lang.NullPointerException'.
Caused by:
java.sql.SQLException: Java exception: ': java.lang.NullPointerException'.
at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.javaException(Unknown Source)
at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.commit(Unknown Source)
at org.apache.commons.dbcp.DelegatingConnection.commit(DelegatingConnection.java:238)
at org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.commit(PoolingDataSource.java:199)
at org.springframework.jdbc.datasource.DataSourceTransactionManager.doCommit(DataSourceTransactionManager.java:245)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:651)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:621)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:146)
at com.aaa.bbb.cccFactory$ddd.add(cccFactory.java:324)
at com.aaa.eee.fff.ggg.reload(ggg.java:44)
at com.aaa.eee.fff.ggg.startup(ggg.java:57)
at com.aaa.eee.fff.Start.startupEee(Start.java:170)
at com.aaa.eee.fff.Start.startup(Start.java:146)
at com.aaa.start.Starter.startup(Starter.java:264)
at com.aaa.start.Main.startup(Main.java:270)
at com.aaa.start.Main.main(Main.java:199)
Caused by: java.sql.SQLException: Java exception: ': java.lang.NullPointerException'.
at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
... 21 more
Caused by: java.lang.NullPointerException
at org.apache.derby.impl.store.access.btree.ControlRow.getControlRowForPage(Unknown Source)
at org.apache.derby.impl.store.access.btree.ControlRow.get(Unknown Source)
at org.apache.derby.impl.store.access.btree.ControlRow.get(Unknown Source)
at org.apache.derby.impl.store.access.btree.BTreePostCommit.purgeRowLevelCommittedDeletes(Unknown Source)
at org.apache.derby.impl.store.access.btree.BTreePostCommit.performWork(Unknown Source)
at org.apache.derby.impl.store.raw.xact.Xact.postTermination(Unknown Source)
at org.apache.derby.impl.store.raw.xact.Xact.completeCommit(Unknown Source)
at org.apache.derby.impl.store.raw.xact.Xact.commit(Unknown Source)
at org.apache.derby.impl.store.raw.xact.Xact.commit(Unknown Source)
at org.apache.derby.impl.store.access.RAMTransaction.commit(Unknown Source)
at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.doCommit(Unknown Source)
at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.userCommit(Unknown Source)
at org.apache.derby.impl.jdbc.TransactionResourceImpl.commit(Unknown Source)
... 15 more
and derby.log:
2008-01-28 08:26:36.148 GMT Thread[Main,5,main] (XID = 1888), (SESSIONID = 2), (DATABASE = directory:myDB), (DRDAID = null), Executing prepared statement: SELECT COUNT(*) FROM ZZZ WHERE nID IS NULL :End prepared statement
2008-01-28 08:26:36.150 GMT Thread[Main,5,main] (XID = 1888), (SESSIONID = 2), (DATABASE = directory:myDB), (DRDAID = null), Committing
2008-01-28 08:26:36.188 GMT Thread[Main,5,main] (XID = 1888), (SESSIONID = 2), (DATABASE = directory:myDB), (DRDAID = null), Cleanup action starting
2008-01-28 08:26:36.188 GMT Thread[Main,5,main] (XID = 1888), (SESSIONID = 2), (DATABASE = directory:myDB), (DRDAID = null), Failed Statement is: null with 2 parameters begin parameter #1: 1 :end parameter begin parameter #2: 1 :end param
eter
java.lang.NullPointerException
at org.apache.derby.impl.store.access.btree.ControlRow.getControlRowForPage(Unknown Source)
at org.apache.derby.impl.store.access.btree.ControlRow.get(Unknown Source)
at org.apache.derby.impl.store.access.btree.ControlRow.get(Unknown Source)
at org.apache.derby.impl.store.access.btree.BTreePostCommit.purgeRowLevelCommittedDeletes(Unknown Source)
at org.apache.derby.impl.store.access.btree.BTreePostCommit.performWork(Unknown Source)
at org.apache.derby.impl.store.raw.xact.Xact.postTermination(Unknown Source)
at org.apache.derby.impl.store.raw.xact.Xact.completeCommit(Unknown Source)
at org.apache.derby.impl.store.raw.xact.Xact.commit(Unknown Source)
at org.apache.derby.impl.store.raw.xact.Xact.commit(Unknown Source)
at org.apache.derby.impl.store.access.RAMTransaction.commit(Unknown Source)
at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.doCommit(Unknown Source)
at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.userCommit(Unknown Source)
at org.apache.derby.impl.jdbc.TransactionResourceImpl.commit(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.commit(Unknown Source)
at org.apache.commons.dbcp.DelegatingConnection.commit(DelegatingConnection.java:238)
at org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.commit(PoolingDataSource.java:199)
at org.springframework.jdbc.datasource.DataSourceTransactionManager.doCommit(DataSourceTransactionManager.java:245)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:651)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:621)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:146)
at com.aaa.bbb.cccFactory$ddd.add(cccFactory.java:324)
at com.aaa.eee.fff.ggg.reload(ggg.java:44)
at com.aaa.eee.fff.ggg.startup(ggg.java:57)
at com.aaa.eee.fff.Start.startupEee(Start.java:170)
at com.aaa.eee.fff.Start.startup(Start.java:146)
at com.aaa.start.Starter.startup(Starter.java:264)
at com.aaa.start.Main.startup(Main.java:270)
at com.aaa.start.Main.main(Main.java:199)
Cleanup action completed
This is a long transaction that has suddenly started throwing a NPE.
Nothing strange happens during the transaction, but on closing I get
the NPE.
If I reorder some of the statements in the transaction (keeping
functional equivalence) the NPE is sometimes not thrown.
(Thanks to Bryan Pendleton on derby-user for pointing me in the direction of DERBY-3216)
> do row level lock space reclamation in btree of indiv rows.
> -----------------------------------------------------------
>
> Key: DERBY-3216
> URL: https://issues.apache.org/jira/browse/DERBY-3216
> Project: Derby
> Issue Type: Bug
> Components: Store
> Affects Versions: 10.3.1.4
> Reporter: Mike Matrigali
> Assignee: Mike Matrigali
> Priority: Minor
> Fix For: 10.3.2.1, 10.4.0.0
>
>
> If you can't get a table level lock for btree space recovery in
> the post commit thread, maybe you should at least reclaim the
> rows on the page while you are at it. Use the same algorithm
> as exists in BTreeController.java. row level shrink is a different
> issue and won't be resolved by this.
> Note there have been reports of "memory" leaks associated with this issue. This is because
> currently if the work can not be done then we just queue it and move on. But in a stress situation
> one may never get the required table lock to shrink the tree and thus the queue just keeps growing.
> Note in many of these cases the app doesn't care if the page merge happens as it is just going to
> insert more rows after the merge.
> Also there is no need for a table level lock for a 1 page index as no merge is actually necessary.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-3216) do row level lock space reclamation
in btree of indiv rows.
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-3216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12545275 ]
Knut Anders Hatlen commented on DERBY-3216:
-------------------------------------------
I noticed in the commit log that a fix had been committed (r597865).
I'm wondering, is the return in this finally clause in purgeRowLevelCommittedDeletes() intentional? It will hide any exception thrown in the try block and make the method return successfully, so it would be good to add a comment explaining why it's there.
+ finally
+ {
+ if (controlRow != null)
+ controlRow.release();
+
+ return;
+ }
> do row level lock space reclamation in btree of indiv rows.
> -----------------------------------------------------------
>
> Key: DERBY-3216
> URL: https://issues.apache.org/jira/browse/DERBY-3216
> Project: Derby
> Issue Type: Bug
> Components: Store
> Affects Versions: 10.3.1.4
> Reporter: Mike Matrigali
> Assignee: Mike Matrigali
> Priority: Minor
>
> If you can't get a table level lock for btree space recovery in
> the post commit thread, maybe you should at least reclaim the
> rows on the page while you are at it. Use the same algorithm
> as exists in BTreeController.java. row level shrink is a different
> issue and won't be resolved by this.
> Note there have been reports of "memory" leaks associated with this issue. This is because
> currently if the work can not be done then we just queue it and move on. But in a stress situation
> one may never get the required table lock to shrink the tree and thus the queue just keeps growing.
> Note in many of these cases the app doesn't care if the page merge happens as it is just going to
> insert more rows after the merge.
> Also there is no need for a table level lock for a 1 page index as no merge is actually necessary.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-3216) do row level lock space reclamation
in btree of indiv rows.
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-3216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12545278 ]
Knut Anders Hatlen commented on DERBY-3216:
-------------------------------------------
In the same method, I see this piece of code:
+ if ((controlRow = ControlRow.get(open_btree, page_number)) == null)
+ return;
As far as I can see, it is impossible that ControlRow.get() returns null, so by returning successfully when controlRow is null, we might be hiding future bugs. Wouldn't it be better to skip the null checking and just let the method fail with an NPE when controlRow is dereferenced?
> do row level lock space reclamation in btree of indiv rows.
> -----------------------------------------------------------
>
> Key: DERBY-3216
> URL: https://issues.apache.org/jira/browse/DERBY-3216
> Project: Derby
> Issue Type: Bug
> Components: Store
> Affects Versions: 10.3.1.4
> Reporter: Mike Matrigali
> Assignee: Mike Matrigali
> Priority: Minor
>
> If you can't get a table level lock for btree space recovery in
> the post commit thread, maybe you should at least reclaim the
> rows on the page while you are at it. Use the same algorithm
> as exists in BTreeController.java. row level shrink is a different
> issue and won't be resolved by this.
> Note there have been reports of "memory" leaks associated with this issue. This is because
> currently if the work can not be done then we just queue it and move on. But in a stress situation
> one may never get the required table lock to shrink the tree and thus the queue just keeps growing.
> Note in many of these cases the app doesn't care if the page merge happens as it is just going to
> insert more rows after the merge.
> Also there is no need for a table level lock for a 1 page index as no merge is actually necessary.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (DERBY-3216) do row level lock space reclamation
in btree of indiv rows.
Posted by "Mike Matrigali (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-3216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mike Matrigali resolved DERBY-3216.
-----------------------------------
Resolution: Fixed
Fix Version/s: 10.4.0.0
10.3.2.0
> do row level lock space reclamation in btree of indiv rows.
> -----------------------------------------------------------
>
> Key: DERBY-3216
> URL: https://issues.apache.org/jira/browse/DERBY-3216
> Project: Derby
> Issue Type: Bug
> Components: Store
> Affects Versions: 10.3.1.4
> Reporter: Mike Matrigali
> Assignee: Mike Matrigali
> Priority: Minor
> Fix For: 10.3.2.0, 10.4.0.0
>
>
> If you can't get a table level lock for btree space recovery in
> the post commit thread, maybe you should at least reclaim the
> rows on the page while you are at it. Use the same algorithm
> as exists in BTreeController.java. row level shrink is a different
> issue and won't be resolved by this.
> Note there have been reports of "memory" leaks associated with this issue. This is because
> currently if the work can not be done then we just queue it and move on. But in a stress situation
> one may never get the required table lock to shrink the tree and thus the queue just keeps growing.
> Note in many of these cases the app doesn't care if the page merge happens as it is just going to
> insert more rows after the merge.
> Also there is no need for a table level lock for a 1 page index as no merge is actually necessary.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-3216) do row level lock space reclamation
in btree of indiv rows.
Posted by "Mike Matrigali (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-3216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12545277 ]
Mike Matrigali commented on DERBY-3216:
---------------------------------------
i will look at moving the return. I was basing the code on code in btreecontroller which also has a return in the finally, didn't really think about it. I'll make the change and run tests and commit.
> do row level lock space reclamation in btree of indiv rows.
> -----------------------------------------------------------
>
> Key: DERBY-3216
> URL: https://issues.apache.org/jira/browse/DERBY-3216
> Project: Derby
> Issue Type: Bug
> Components: Store
> Affects Versions: 10.3.1.4
> Reporter: Mike Matrigali
> Assignee: Mike Matrigali
> Priority: Minor
>
> If you can't get a table level lock for btree space recovery in
> the post commit thread, maybe you should at least reclaim the
> rows on the page while you are at it. Use the same algorithm
> as exists in BTreeController.java. row level shrink is a different
> issue and won't be resolved by this.
> Note there have been reports of "memory" leaks associated with this issue. This is because
> currently if the work can not be done then we just queue it and move on. But in a stress situation
> one may never get the required table lock to shrink the tree and thus the queue just keeps growing.
> Note in many of these cases the app doesn't care if the page merge happens as it is just going to
> insert more rows after the merge.
> Also there is no need for a table level lock for a 1 page index as no merge is actually necessary.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.