You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by "Mike Matrigali (JIRA)" <ji...@apache.org> on 2011/06/01 19:50:47 UTC
[jira] [Created] (DERBY-5258) btree post commit releases latch
before committing/aborting purges, possibly allowing other operation on
page
btree post commit releases latch before committing/aborting purges, possibly allowing other operation on page
-------------------------------------------------------------------------------------------------------------
Key: DERBY-5258
URL: https://issues.apache.org/jira/browse/DERBY-5258
Project: Derby
Issue Type: Bug
Components: Store
Affects Versions: 10.8.1.2, 10.7.1.1, 10.6.1.0, 10.5.3.0, 10.4.2.0, 10.3.3.0, 10.2.2.0, 10.1.3.1
Reporter: Mike Matrigali
Assignee: Mike Matrigali
>From code inspection found the following problem. BTreePostCommit.purgeCommittedDeletes gives up the latch in it's finally block, before the internal transaction is
committed. The transaction is committed no sync upon return from this routine leaving a very small window when some other thread could get latch on the page and
perform operations on the page.
This can be a problem if for some reason the internal transaction is never committed. Purges actually return space to the page, unlike deletes. In order to backout the
purges one must add the rows back, taking up space on the page. If another tranaction comes in before the internal transaction is committed and does inserts there may
be no space for the backout of the purges. This is why normal delete processing only sets flags on the rows and purge processing is handled differently.
I found this problem while debugging a database submitted as part of DERBY-5248. I believe this issue can cause the problem there, but since we have no repro have
decided to create a new issue to target this specific problem/solution. Later can close the other issue if it can never be reproduce after the fix. In DEBY-5248 there
are purges without a commit followed immediated by an insert in another transaction that is commited and the purge transaction is never committed. On recovery the
system tries to abort the internal transaction and eventually trashes the page when it does not actually have enough space to abort the purge. See that issue for more
detail.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (DERBY-5258) btree post commit releases latch
before committing/aborting purges, possibly allowing other operation on
page
Posted by "Mike Matrigali (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mike Matrigali updated DERBY-5258:
----------------------------------
Affects Version/s: (was: 10.4.2.1)
(was: 10.2.2.0)
(was: 10.1.3.1)
fixing Affects version field. This bug was introduced in 10.3 when row level purging of btree rows during post commit was added.
Earlier releases were not affected.
> btree post commit releases latch before committing/aborting purges, possibly allowing other operation on page
> -------------------------------------------------------------------------------------------------------------
>
> Key: DERBY-5258
> URL: https://issues.apache.org/jira/browse/DERBY-5258
> Project: Derby
> Issue Type: Bug
> Components: Store
> Affects Versions: 10.3.3.0, 10.4.2.0, 10.5.3.0, 10.6.1.0, 10.7.1.1, 10.8.1.2
> Reporter: Mike Matrigali
> Assignee: Mike Matrigali
> Fix For: 10.5.3.2, 10.6.2.3, 10.7.1.4, 10.8.1.4, 10.9.0.0
>
> Attachments: DERBY-5258_diff.txt
>
>
> From code inspection found the following problem. BTreePostCommit.purgeCommittedDeletes gives up the latch in it's finally block, before the internal transaction is
> committed. The transaction is committed no sync upon return from this routine leaving a very small window when some other thread could get latch on the page and
> perform operations on the page.
> This can be a problem if for some reason the internal transaction is never committed. Purges actually return space to the page, unlike deletes. In order to backout the
> purges one must add the rows back, taking up space on the page. If another tranaction comes in before the internal transaction is committed and does inserts there may
> be no space for the backout of the purges. This is why normal delete processing only sets flags on the rows and purge processing is handled differently.
> I found this problem while debugging a database submitted as part of DERBY-5248. I believe this issue can cause the problem there, but since we have no repro have
> decided to create a new issue to target this specific problem/solution. Later can close the other issue if it can never be reproduce after the fix. In DEBY-5248 there
> are purges without a commit followed immediated by an insert in another transaction that is commited and the purge transaction is never committed. On recovery the
> system tries to abort the internal transaction and eventually trashes the page when it does not actually have enough space to abort the purge. See that issue for more
> detail.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (DERBY-5258) btree post commit releases latch
before committing/aborting purges, possibly allowing other operation on
page
Posted by "Mike Matrigali (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mike Matrigali resolved DERBY-5258.
-----------------------------------
Resolution: Fixed
Found issue by code inspection, fixed and backported to all affectected branches. 10.5 branch is oldest branch affected by this issue.
> btree post commit releases latch before committing/aborting purges, possibly allowing other operation on page
> -------------------------------------------------------------------------------------------------------------
>
> Key: DERBY-5258
> URL: https://issues.apache.org/jira/browse/DERBY-5258
> Project: Derby
> Issue Type: Bug
> Components: Store
> Affects Versions: 10.3.3.0, 10.4.2.0, 10.5.3.0, 10.6.1.0, 10.7.1.1, 10.8.1.2
> Reporter: Mike Matrigali
> Assignee: Mike Matrigali
> Fix For: 10.5.3.2, 10.6.2.3, 10.7.1.4, 10.8.1.5, 10.9.0.0
>
> Attachments: DERBY-5258_diff.txt
>
>
> From code inspection found the following problem. BTreePostCommit.purgeCommittedDeletes gives up the latch in it's finally block, before the internal transaction is
> committed. The transaction is committed no sync upon return from this routine leaving a very small window when some other thread could get latch on the page and
> perform operations on the page.
> This can be a problem if for some reason the internal transaction is never committed. Purges actually return space to the page, unlike deletes. In order to backout the
> purges one must add the rows back, taking up space on the page. If another tranaction comes in before the internal transaction is committed and does inserts there may
> be no space for the backout of the purges. This is why normal delete processing only sets flags on the rows and purge processing is handled differently.
> I found this problem while debugging a database submitted as part of DERBY-5248. I believe this issue can cause the problem there, but since we have no repro have
> decided to create a new issue to target this specific problem/solution. Later can close the other issue if it can never be reproduce after the fix. In DEBY-5248 there
> are purges without a commit followed immediated by an insert in another transaction that is commited and the purge transaction is never committed. On recovery the
> system tries to abort the internal transaction and eventually trashes the page when it does not actually have enough space to abort the purge. See that issue for more
> detail.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (DERBY-5258) btree post commit releases latch
before committing/aborting purges, possibly allowing other operation on
page
Posted by "Tim Wu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13045812#comment-13045812 ]
Tim Wu commented on DERBY-5258:
-------------------------------
Hi Mike, Any chance this fix can be merged back to 10.8?
> btree post commit releases latch before committing/aborting purges, possibly allowing other operation on page
> -------------------------------------------------------------------------------------------------------------
>
> Key: DERBY-5258
> URL: https://issues.apache.org/jira/browse/DERBY-5258
> Project: Derby
> Issue Type: Bug
> Components: Store
> Affects Versions: 10.1.3.1, 10.2.2.0, 10.3.3.0, 10.4.2.0, 10.5.3.0, 10.6.1.0, 10.7.1.1, 10.8.1.2
> Reporter: Mike Matrigali
> Assignee: Mike Matrigali
> Fix For: 10.9.0.0
>
> Attachments: DERBY-5258_diff.txt
>
>
> From code inspection found the following problem. BTreePostCommit.purgeCommittedDeletes gives up the latch in it's finally block, before the internal transaction is
> committed. The transaction is committed no sync upon return from this routine leaving a very small window when some other thread could get latch on the page and
> perform operations on the page.
> This can be a problem if for some reason the internal transaction is never committed. Purges actually return space to the page, unlike deletes. In order to backout the
> purges one must add the rows back, taking up space on the page. If another tranaction comes in before the internal transaction is committed and does inserts there may
> be no space for the backout of the purges. This is why normal delete processing only sets flags on the rows and purge processing is handled differently.
> I found this problem while debugging a database submitted as part of DERBY-5248. I believe this issue can cause the problem there, but since we have no repro have
> decided to create a new issue to target this specific problem/solution. Later can close the other issue if it can never be reproduce after the fix. In DEBY-5248 there
> are purges without a commit followed immediated by an insert in another transaction that is commited and the purge transaction is never committed. On recovery the
> system tries to abort the internal transaction and eventually trashes the page when it does not actually have enough space to abort the purge. See that issue for more
> detail.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (DERBY-5258) btree post commit releases latch
before committing/aborting purges, possibly allowing other operation on
page
Posted by "Mike Matrigali (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mike Matrigali updated DERBY-5258:
----------------------------------
Fix Version/s: 10.9.0.0
> btree post commit releases latch before committing/aborting purges, possibly allowing other operation on page
> -------------------------------------------------------------------------------------------------------------
>
> Key: DERBY-5258
> URL: https://issues.apache.org/jira/browse/DERBY-5258
> Project: Derby
> Issue Type: Bug
> Components: Store
> Affects Versions: 10.1.3.1, 10.2.2.0, 10.3.3.0, 10.4.2.0, 10.5.3.0, 10.6.1.0, 10.7.1.1, 10.8.1.2
> Reporter: Mike Matrigali
> Assignee: Mike Matrigali
> Fix For: 10.9.0.0
>
> Attachments: DERBY-5258_diff.txt
>
>
> From code inspection found the following problem. BTreePostCommit.purgeCommittedDeletes gives up the latch in it's finally block, before the internal transaction is
> committed. The transaction is committed no sync upon return from this routine leaving a very small window when some other thread could get latch on the page and
> perform operations on the page.
> This can be a problem if for some reason the internal transaction is never committed. Purges actually return space to the page, unlike deletes. In order to backout the
> purges one must add the rows back, taking up space on the page. If another tranaction comes in before the internal transaction is committed and does inserts there may
> be no space for the backout of the purges. This is why normal delete processing only sets flags on the rows and purge processing is handled differently.
> I found this problem while debugging a database submitted as part of DERBY-5248. I believe this issue can cause the problem there, but since we have no repro have
> decided to create a new issue to target this specific problem/solution. Later can close the other issue if it can never be reproduce after the fix. In DEBY-5248 there
> are purges without a commit followed immediated by an insert in another transaction that is commited and the purge transaction is never committed. On recovery the
> system tries to abort the internal transaction and eventually trashes the page when it does not actually have enough space to abort the purge. See that issue for more
> detail.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (DERBY-5258) btree post commit releases latch
before committing/aborting purges, possibly allowing other operation on
page
Posted by "Bryan Pendleton (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13044417#comment-13044417 ]
Bryan Pendleton commented on DERBY-5258:
----------------------------------------
Wow! Great find, Mike! Your description is quite clear and your theory makes sense to me.
Your detective work is wonderful to read, thanks very much for taking the time to describe
the sequence of actions that lead to this case.
The patch has a few whitespace issues, I think, but other than that it looked fine to me.
I think that the comment could read "The commit will clear the latch" rather than
"The commit should clear the latch", as the "should" doesn't quite sound right.
> btree post commit releases latch before committing/aborting purges, possibly allowing other operation on page
> -------------------------------------------------------------------------------------------------------------
>
> Key: DERBY-5258
> URL: https://issues.apache.org/jira/browse/DERBY-5258
> Project: Derby
> Issue Type: Bug
> Components: Store
> Affects Versions: 10.1.3.1, 10.2.2.0, 10.3.3.0, 10.4.2.0, 10.5.3.0, 10.6.1.0, 10.7.1.1, 10.8.1.2
> Reporter: Mike Matrigali
> Assignee: Mike Matrigali
> Attachments: DERBY-5258_diff.txt
>
>
> From code inspection found the following problem. BTreePostCommit.purgeCommittedDeletes gives up the latch in it's finally block, before the internal transaction is
> committed. The transaction is committed no sync upon return from this routine leaving a very small window when some other thread could get latch on the page and
> perform operations on the page.
> This can be a problem if for some reason the internal transaction is never committed. Purges actually return space to the page, unlike deletes. In order to backout the
> purges one must add the rows back, taking up space on the page. If another tranaction comes in before the internal transaction is committed and does inserts there may
> be no space for the backout of the purges. This is why normal delete processing only sets flags on the rows and purge processing is handled differently.
> I found this problem while debugging a database submitted as part of DERBY-5248. I believe this issue can cause the problem there, but since we have no repro have
> decided to create a new issue to target this specific problem/solution. Later can close the other issue if it can never be reproduce after the fix. In DEBY-5248 there
> are purges without a commit followed immediated by an insert in another transaction that is commited and the purge transaction is never committed. On recovery the
> system tries to abort the internal transaction and eventually trashes the page when it does not actually have enough space to abort the purge. See that issue for more
> detail.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (DERBY-5258) btree post commit releases latch
before committing/aborting purges, possibly allowing other operation on
page
Posted by "Mike Matrigali (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13044983#comment-13044983 ]
Mike Matrigali commented on DERBY-5258:
---------------------------------------
All the tests passed, and I integrated Bryan's suggested comment changes. Committed to trunk:
Sending java\engine\org\apache\derby\impl\store\access\btree\BTreePostCommit.java
Sending java\engine\org\apache\derby\impl\store\raw\data\BasePage.java
Sending java\engine\org\apache\derby\impl\store\raw\data\StoredPage.java
Transmitting file data ...
Committed revision 1132711.
> btree post commit releases latch before committing/aborting purges, possibly allowing other operation on page
> -------------------------------------------------------------------------------------------------------------
>
> Key: DERBY-5258
> URL: https://issues.apache.org/jira/browse/DERBY-5258
> Project: Derby
> Issue Type: Bug
> Components: Store
> Affects Versions: 10.1.3.1, 10.2.2.0, 10.3.3.0, 10.4.2.0, 10.5.3.0, 10.6.1.0, 10.7.1.1, 10.8.1.2
> Reporter: Mike Matrigali
> Assignee: Mike Matrigali
> Fix For: 10.9.0.0
>
> Attachments: DERBY-5258_diff.txt
>
>
> From code inspection found the following problem. BTreePostCommit.purgeCommittedDeletes gives up the latch in it's finally block, before the internal transaction is
> committed. The transaction is committed no sync upon return from this routine leaving a very small window when some other thread could get latch on the page and
> perform operations on the page.
> This can be a problem if for some reason the internal transaction is never committed. Purges actually return space to the page, unlike deletes. In order to backout the
> purges one must add the rows back, taking up space on the page. If another tranaction comes in before the internal transaction is committed and does inserts there may
> be no space for the backout of the purges. This is why normal delete processing only sets flags on the rows and purge processing is handled differently.
> I found this problem while debugging a database submitted as part of DERBY-5248. I believe this issue can cause the problem there, but since we have no repro have
> decided to create a new issue to target this specific problem/solution. Later can close the other issue if it can never be reproduce after the fix. In DEBY-5248 there
> are purges without a commit followed immediated by an insert in another transaction that is commited and the purge transaction is never committed. On recovery the
> system tries to abort the internal transaction and eventually trashes the page when it does not actually have enough space to abort the purge. See that issue for more
> detail.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (DERBY-5258) btree post commit releases latch
before committing/aborting purges, possibly allowing other operation on
page
Posted by "Mike Matrigali (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mike Matrigali updated DERBY-5258:
----------------------------------
Fix Version/s: 10.5.3.2
backported from trunk to 10.5 branch with minor conflict fix:
s105_ibm16:18>svn commit
Sending java\engine\org\apache\derby\impl\store\access\btree\BTreePostCommit.java
Sending java\engine\org\apache\derby\impl\store\raw\data\BasePage.java
Sending java\engine\org\apache\derby\impl\store\raw\data\StoredPage.java
Transmitting file data ...
Committed revision 1134103.
> btree post commit releases latch before committing/aborting purges, possibly allowing other operation on page
> -------------------------------------------------------------------------------------------------------------
>
> Key: DERBY-5258
> URL: https://issues.apache.org/jira/browse/DERBY-5258
> Project: Derby
> Issue Type: Bug
> Components: Store
> Affects Versions: 10.1.3.1, 10.2.2.0, 10.3.3.0, 10.4.2.0, 10.5.3.0, 10.6.1.0, 10.7.1.1, 10.8.1.2
> Reporter: Mike Matrigali
> Assignee: Mike Matrigali
> Fix For: 10.5.3.2, 10.6.2.3, 10.7.1.4, 10.8.1.4, 10.9.0.0
>
> Attachments: DERBY-5258_diff.txt
>
>
> From code inspection found the following problem. BTreePostCommit.purgeCommittedDeletes gives up the latch in it's finally block, before the internal transaction is
> committed. The transaction is committed no sync upon return from this routine leaving a very small window when some other thread could get latch on the page and
> perform operations on the page.
> This can be a problem if for some reason the internal transaction is never committed. Purges actually return space to the page, unlike deletes. In order to backout the
> purges one must add the rows back, taking up space on the page. If another tranaction comes in before the internal transaction is committed and does inserts there may
> be no space for the backout of the purges. This is why normal delete processing only sets flags on the rows and purge processing is handled differently.
> I found this problem while debugging a database submitted as part of DERBY-5248. I believe this issue can cause the problem there, but since we have no repro have
> decided to create a new issue to target this specific problem/solution. Later can close the other issue if it can never be reproduce after the fix. In DEBY-5248 there
> are purges without a commit followed immediated by an insert in another transaction that is commited and the purge transaction is never committed. On recovery the
> system tries to abort the internal transaction and eventually trashes the page when it does not actually have enough space to abort the purge. See that issue for more
> detail.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (DERBY-5258) btree post commit releases latch
before committing/aborting purges, possibly allowing other operation on
page
Posted by "Mike Matrigali (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mike Matrigali updated DERBY-5258:
----------------------------------
Fix Version/s: 10.7.1.4
backported the fix from trunk to 10.7 branch.
s107_ibm16:5>svn commit
Sending java\engine\org\apache\derby\impl\store\access\btree\BTreePostCommit.java
Sending java\engine\org\apache\derby\impl\store\raw\data\BasePage.java
Sending java\engine\org\apache\derby\impl\store\raw\data\StoredPage.java
Transmitting file data ...
Committed revision 1134092.
> btree post commit releases latch before committing/aborting purges, possibly allowing other operation on page
> -------------------------------------------------------------------------------------------------------------
>
> Key: DERBY-5258
> URL: https://issues.apache.org/jira/browse/DERBY-5258
> Project: Derby
> Issue Type: Bug
> Components: Store
> Affects Versions: 10.1.3.1, 10.2.2.0, 10.3.3.0, 10.4.2.0, 10.5.3.0, 10.6.1.0, 10.7.1.1, 10.8.1.2
> Reporter: Mike Matrigali
> Assignee: Mike Matrigali
> Fix For: 10.7.1.4, 10.8.1.4, 10.9.0.0
>
> Attachments: DERBY-5258_diff.txt
>
>
> From code inspection found the following problem. BTreePostCommit.purgeCommittedDeletes gives up the latch in it's finally block, before the internal transaction is
> committed. The transaction is committed no sync upon return from this routine leaving a very small window when some other thread could get latch on the page and
> perform operations on the page.
> This can be a problem if for some reason the internal transaction is never committed. Purges actually return space to the page, unlike deletes. In order to backout the
> purges one must add the rows back, taking up space on the page. If another tranaction comes in before the internal transaction is committed and does inserts there may
> be no space for the backout of the purges. This is why normal delete processing only sets flags on the rows and purge processing is handled differently.
> I found this problem while debugging a database submitted as part of DERBY-5248. I believe this issue can cause the problem there, but since we have no repro have
> decided to create a new issue to target this specific problem/solution. Later can close the other issue if it can never be reproduce after the fix. In DEBY-5248 there
> are purges without a commit followed immediated by an insert in another transaction that is commited and the purge transaction is never committed. On recovery the
> system tries to abort the internal transaction and eventually trashes the page when it does not actually have enough space to abort the purge. See that issue for more
> detail.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (DERBY-5258) btree post commit releases latch
before committing/aborting purges, possibly allowing other operation on
page
Posted by "Mike Matrigali (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13046052#comment-13046052 ]
Mike Matrigali commented on DERBY-5258:
---------------------------------------
i plan to backport once I verify that tests passed across all nightly's on trunk and I run tests on 10.8.
> btree post commit releases latch before committing/aborting purges, possibly allowing other operation on page
> -------------------------------------------------------------------------------------------------------------
>
> Key: DERBY-5258
> URL: https://issues.apache.org/jira/browse/DERBY-5258
> Project: Derby
> Issue Type: Bug
> Components: Store
> Affects Versions: 10.1.3.1, 10.2.2.0, 10.3.3.0, 10.4.2.0, 10.5.3.0, 10.6.1.0, 10.7.1.1, 10.8.1.2
> Reporter: Mike Matrigali
> Assignee: Mike Matrigali
> Fix For: 10.9.0.0
>
> Attachments: DERBY-5258_diff.txt
>
>
> From code inspection found the following problem. BTreePostCommit.purgeCommittedDeletes gives up the latch in it's finally block, before the internal transaction is
> committed. The transaction is committed no sync upon return from this routine leaving a very small window when some other thread could get latch on the page and
> perform operations on the page.
> This can be a problem if for some reason the internal transaction is never committed. Purges actually return space to the page, unlike deletes. In order to backout the
> purges one must add the rows back, taking up space on the page. If another tranaction comes in before the internal transaction is committed and does inserts there may
> be no space for the backout of the purges. This is why normal delete processing only sets flags on the rows and purge processing is handled differently.
> I found this problem while debugging a database submitted as part of DERBY-5248. I believe this issue can cause the problem there, but since we have no repro have
> decided to create a new issue to target this specific problem/solution. Later can close the other issue if it can never be reproduce after the fix. In DEBY-5248 there
> are purges without a commit followed immediated by an insert in another transaction that is commited and the purge transaction is never committed. On recovery the
> system tries to abort the internal transaction and eventually trashes the page when it does not actually have enough space to abort the purge. See that issue for more
> detail.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (DERBY-5258) btree post commit releases latch
before committing/aborting purges, possibly allowing other operation on
page
Posted by "Mike Matrigali (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mike Matrigali updated DERBY-5258:
----------------------------------
Affects Version/s: 10.4.2.1
backported fix from trunk to 10.4 branch, required some manual merging of conflicts.
s104_jdk16:18>svn commit
Sending java\engine\org\apache\derby\impl\store\access\btree\BTreePostCommit.java
Sending java\engine\org\apache\derby\impl\store\raw\data\BasePage.java
Sending java\engine\org\apache\derby\impl\store\raw\data\StoredPage.java
Transmitting file data ...
Committed revision 1135720.
> btree post commit releases latch before committing/aborting purges, possibly allowing other operation on page
> -------------------------------------------------------------------------------------------------------------
>
> Key: DERBY-5258
> URL: https://issues.apache.org/jira/browse/DERBY-5258
> Project: Derby
> Issue Type: Bug
> Components: Store
> Affects Versions: 10.1.3.1, 10.2.2.0, 10.3.3.0, 10.4.2.0, 10.4.2.1, 10.5.3.0, 10.6.1.0, 10.7.1.1, 10.8.1.2
> Reporter: Mike Matrigali
> Assignee: Mike Matrigali
> Fix For: 10.5.3.2, 10.6.2.3, 10.7.1.4, 10.8.1.4, 10.9.0.0
>
> Attachments: DERBY-5258_diff.txt
>
>
> From code inspection found the following problem. BTreePostCommit.purgeCommittedDeletes gives up the latch in it's finally block, before the internal transaction is
> committed. The transaction is committed no sync upon return from this routine leaving a very small window when some other thread could get latch on the page and
> perform operations on the page.
> This can be a problem if for some reason the internal transaction is never committed. Purges actually return space to the page, unlike deletes. In order to backout the
> purges one must add the rows back, taking up space on the page. If another tranaction comes in before the internal transaction is committed and does inserts there may
> be no space for the backout of the purges. This is why normal delete processing only sets flags on the rows and purge processing is handled differently.
> I found this problem while debugging a database submitted as part of DERBY-5248. I believe this issue can cause the problem there, but since we have no repro have
> decided to create a new issue to target this specific problem/solution. Later can close the other issue if it can never be reproduce after the fix. In DEBY-5248 there
> are purges without a commit followed immediated by an insert in another transaction that is commited and the purge transaction is never committed. On recovery the
> system tries to abort the internal transaction and eventually trashes the page when it does not actually have enough space to abort the purge. See that issue for more
> detail.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (DERBY-5258) btree post commit releases latch
before committing/aborting purges, possibly allowing other operation on
page
Posted by "Mike Matrigali (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mike Matrigali updated DERBY-5258:
----------------------------------
Fix Version/s: 10.6.2.3
back ported to 10.6 branch, with minor conflict fix:
s106_ibm16:31>svn commit
Sending java\engine\org\apache\derby\impl\store\access\btree\BTreePostCommit.java
Sending java\engine\org\apache\derby\impl\store\raw\data\BasePage.java
Sending java\engine\org\apache\derby\impl\store\raw\data\StoredPage.java
Transmitting file data ...
Committed revision 1134098.
> btree post commit releases latch before committing/aborting purges, possibly allowing other operation on page
> -------------------------------------------------------------------------------------------------------------
>
> Key: DERBY-5258
> URL: https://issues.apache.org/jira/browse/DERBY-5258
> Project: Derby
> Issue Type: Bug
> Components: Store
> Affects Versions: 10.1.3.1, 10.2.2.0, 10.3.3.0, 10.4.2.0, 10.5.3.0, 10.6.1.0, 10.7.1.1, 10.8.1.2
> Reporter: Mike Matrigali
> Assignee: Mike Matrigali
> Fix For: 10.6.2.3, 10.7.1.4, 10.8.1.4, 10.9.0.0
>
> Attachments: DERBY-5258_diff.txt
>
>
> From code inspection found the following problem. BTreePostCommit.purgeCommittedDeletes gives up the latch in it's finally block, before the internal transaction is
> committed. The transaction is committed no sync upon return from this routine leaving a very small window when some other thread could get latch on the page and
> perform operations on the page.
> This can be a problem if for some reason the internal transaction is never committed. Purges actually return space to the page, unlike deletes. In order to backout the
> purges one must add the rows back, taking up space on the page. If another tranaction comes in before the internal transaction is committed and does inserts there may
> be no space for the backout of the purges. This is why normal delete processing only sets flags on the rows and purge processing is handled differently.
> I found this problem while debugging a database submitted as part of DERBY-5248. I believe this issue can cause the problem there, but since we have no repro have
> decided to create a new issue to target this specific problem/solution. Later can close the other issue if it can never be reproduce after the fix. In DEBY-5248 there
> are purges without a commit followed immediated by an insert in another transaction that is commited and the purge transaction is never committed. On recovery the
> system tries to abort the internal transaction and eventually trashes the page when it does not actually have enough space to abort the purge. See that issue for more
> detail.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (DERBY-5258) btree post commit releases latch
before committing/aborting purges, possibly allowing other operation on
page
Posted by "Mike Matrigali (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mike Matrigali updated DERBY-5258:
----------------------------------
Attachment: DERBY-5258_diff.txt
Attaching proposed fix. This has not passed tests yet.
I have not been able to reproduce the problem in my environment so the fix is just from code inspection. One change from previous comments is that this fix is to
purgeRowLevelCommittedDeletes(). The purgeCommittedDeletes() routine is ok, as it holds a table
level exclusive lock until end of transaction.
I will run full set of tests before committing to trunk.
I will try for another day to reproduce, but if I can't will probably go ahead and check in to trunk unless anyone
feels I should not.
> btree post commit releases latch before committing/aborting purges, possibly allowing other operation on page
> -------------------------------------------------------------------------------------------------------------
>
> Key: DERBY-5258
> URL: https://issues.apache.org/jira/browse/DERBY-5258
> Project: Derby
> Issue Type: Bug
> Components: Store
> Affects Versions: 10.1.3.1, 10.2.2.0, 10.3.3.0, 10.4.2.0, 10.5.3.0, 10.6.1.0, 10.7.1.1, 10.8.1.2
> Reporter: Mike Matrigali
> Assignee: Mike Matrigali
> Attachments: DERBY-5258_diff.txt
>
>
> From code inspection found the following problem. BTreePostCommit.purgeCommittedDeletes gives up the latch in it's finally block, before the internal transaction is
> committed. The transaction is committed no sync upon return from this routine leaving a very small window when some other thread could get latch on the page and
> perform operations on the page.
> This can be a problem if for some reason the internal transaction is never committed. Purges actually return space to the page, unlike deletes. In order to backout the
> purges one must add the rows back, taking up space on the page. If another tranaction comes in before the internal transaction is committed and does inserts there may
> be no space for the backout of the purges. This is why normal delete processing only sets flags on the rows and purge processing is handled differently.
> I found this problem while debugging a database submitted as part of DERBY-5248. I believe this issue can cause the problem there, but since we have no repro have
> decided to create a new issue to target this specific problem/solution. Later can close the other issue if it can never be reproduce after the fix. In DEBY-5248 there
> are purges without a commit followed immediated by an insert in another transaction that is commited and the purge transaction is never committed. On recovery the
> system tries to abort the internal transaction and eventually trashes the page when it does not actually have enough space to abort the purge. See that issue for more
> detail.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (DERBY-5258) btree post commit releases latch
before committing/aborting purges, possibly allowing other operation on
page
Posted by "Mike Matrigali (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mike Matrigali updated DERBY-5258:
----------------------------------
Fix Version/s: 10.8.1.4
backported change from trunk to 10.8 branch:
s108_ibm16:11>svn commit
Sending java\engine\org\apache\derby\impl\store\access\btree\BTreePostCommit.java
Sending java\engine\org\apache\derby\impl\store\raw\data\BasePage.java
Sending java\engine\org\apache\derby\impl\store\raw\data\StoredPage.java
Transmitting file data ...
Committed revision 1133470.
> btree post commit releases latch before committing/aborting purges, possibly allowing other operation on page
> -------------------------------------------------------------------------------------------------------------
>
> Key: DERBY-5258
> URL: https://issues.apache.org/jira/browse/DERBY-5258
> Project: Derby
> Issue Type: Bug
> Components: Store
> Affects Versions: 10.1.3.1, 10.2.2.0, 10.3.3.0, 10.4.2.0, 10.5.3.0, 10.6.1.0, 10.7.1.1, 10.8.1.2
> Reporter: Mike Matrigali
> Assignee: Mike Matrigali
> Fix For: 10.8.1.4, 10.9.0.0
>
> Attachments: DERBY-5258_diff.txt
>
>
> From code inspection found the following problem. BTreePostCommit.purgeCommittedDeletes gives up the latch in it's finally block, before the internal transaction is
> committed. The transaction is committed no sync upon return from this routine leaving a very small window when some other thread could get latch on the page and
> perform operations on the page.
> This can be a problem if for some reason the internal transaction is never committed. Purges actually return space to the page, unlike deletes. In order to backout the
> purges one must add the rows back, taking up space on the page. If another tranaction comes in before the internal transaction is committed and does inserts there may
> be no space for the backout of the purges. This is why normal delete processing only sets flags on the rows and purge processing is handled differently.
> I found this problem while debugging a database submitted as part of DERBY-5248. I believe this issue can cause the problem there, but since we have no repro have
> decided to create a new issue to target this specific problem/solution. Later can close the other issue if it can never be reproduce after the fix. In DEBY-5248 there
> are purges without a commit followed immediated by an insert in another transaction that is commited and the purge transaction is never committed. On recovery the
> system tries to abort the internal transaction and eventually trashes the page when it does not actually have enough space to abort the purge. See that issue for more
> detail.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira