You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by "Knut Anders Hatlen (JIRA)" <ji...@apache.org> on 2007/10/03 10:56:50 UTC
[jira] Created: (DERBY-3099) Possible bug in interaction with
buffer manager causing pages not to be freed on rollback to savepoint
Possible bug in interaction with buffer manager causing pages not to be freed on rollback to savepoint
------------------------------------------------------------------------------------------------------
Key: DERBY-3099
URL: https://issues.apache.org/jira/browse/DERBY-3099
Project: Derby
Issue Type: Bug
Components: Services, Store
Affects Versions: 10.4.0.0
Reporter: Knut Anders Hatlen
I noticed this strange behaviour when I was working on DERBY-2911. It seems like the result of test case P042 in unit/T_RawStoreFactory.unit is dependent on the actual contents of the buffer manager (which pages have been evicted, which free entries have been reused and so on). I'm not sure if this is a bug in the test or somewhere in the code, or if this is expected behaviour, but it sounds a bit suspicious.
For instance, commenting out all the test cases preceeding P042 makes P042 fail, even though P042 creates a new container so that it should not be affected by any of the previous test cases. Also, commenting out a space optimization in Clock.findFreeItem() so that freed entries are not reused except when rotateClock() is called on a full cache to find a victim, causes the test case to fail. A third way to make it fail, is to vary the scan direction when looking for a free entry to reuse in the new buffer manager (ConcurrentCache). If the scan is disabled or walks the clock from the beginning to the end, the test fails, but if the clock is scanned backwards, it passes.
The code that fails in the test, is
c = t_util.t_openContainer(t, segment, cid, true);
Page checkNextPage = t_util.t_addPage(c);
if (checkNextPage.getPageNumber() == nextPageNumber)
throw T_Fail.testFailMsg(
"expect some pages to be freed by update rollback");
The expected page number is 2, and the actual page number is 7.
Before this, a large row has been inserted on page 1 and overflows to page 2, 3, 4, 5 and 6. Also, page 7 has been added manually before all the updates were rolled back so that the pages from 2 up to 7 should be unallocated. Page 7 is then added and removed, and the transaction is committed. After reopening the container, the test expects the pages from 2 up to 7 to be free, and that t_addPage() should allocate page number 2.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: [jira] Updated: (DERBY-3099) Possible bug in interaction with buffer manager causing pages not to be freed on rollback to savepoint
Posted by Myrna van Lunteren <m....@gmail.com>.
On 10/10/07, Knut Anders Hatlen (JIRA) <ji...@apache.org> wrote:
>
> [ https://issues.apache.org/jira/browse/DERBY-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
>
> Knut Anders Hatlen updated DERBY-3099:
> --------------------------------------
>
> Affects Version/s: 10.3.1.4
>
> The bug is also present on the 10.3 branch (I applied the assert.diff patch and got the expected assert failures when I tried to connect to a database). Should the fix be backported, or is it safer just to leave it as it is since there are no known user-visible consequences?
>
I'd say, put it in 10.3. A fix, is a fix, even if it's not apparent to
(end-)users.
Myrna
[jira] Updated: (DERBY-3099) Possible bug in interaction with
buffer manager causing pages not to be freed on rollback to savepoint
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Knut Anders Hatlen updated DERBY-3099:
--------------------------------------
Fix Version/s: 10.3.1.5
Committed to 10.3 with revision 583956.
> Possible bug in interaction with buffer manager causing pages not to be freed on rollback to savepoint
> ------------------------------------------------------------------------------------------------------
>
> Key: DERBY-3099
> URL: https://issues.apache.org/jira/browse/DERBY-3099
> Project: Derby
> Issue Type: Bug
> Components: Services, Store
> Affects Versions: 10.3.1.4, 10.4.0.0
> Reporter: Knut Anders Hatlen
> Assignee: Knut Anders Hatlen
> Fix For: 10.3.1.5, 10.4.0.0
>
> Attachments: assert.diff, d3099-1.diff, failfast.diff, initSpace.diff
>
>
> I noticed this strange behaviour when I was working on DERBY-2911. It seems like the result of test case P042 in unit/T_RawStoreFactory.unit is dependent on the actual contents of the buffer manager (which pages have been evicted, which free entries have been reused and so on). I'm not sure if this is a bug in the test or somewhere in the code, or if this is expected behaviour, but it sounds a bit suspicious.
> For instance, commenting out all the test cases preceeding P042 makes P042 fail, even though P042 creates a new container so that it should not be affected by any of the previous test cases. Also, commenting out a space optimization in Clock.findFreeItem() so that freed entries are not reused except when rotateClock() is called on a full cache to find a victim, causes the test case to fail. A third way to make it fail, is to vary the scan direction when looking for a free entry to reuse in the new buffer manager (ConcurrentCache). If the scan is disabled or walks the clock from the beginning to the end, the test fails, but if the clock is scanned backwards, it passes.
> The code that fails in the test, is
> c = t_util.t_openContainer(t, segment, cid, true);
> Page checkNextPage = t_util.t_addPage(c);
> if (checkNextPage.getPageNumber() == nextPageNumber)
> throw T_Fail.testFailMsg(
> "expect some pages to be freed by update rollback");
> The expected page number is 2, and the actual page number is 7.
> Before this, a large row has been inserted on page 1 and overflows to page 2, 3, 4, 5 and 6. Also, page 7 has been added manually before all the updates were rolled back so that the pages from 2 up to 7 should be unallocated. Page 7 is then added and removed, and the transaction is committed. After reopening the container, the test expects the pages from 2 up to 7 to be free, and that t_addPage() should allocate page number 2.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-3099) Possible bug in interaction with
buffer manager causing pages not to be freed on rollback to savepoint
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Knut Anders Hatlen updated DERBY-3099:
--------------------------------------
Attachment: d3099-1.diff
The patch d3099-1.diff combines initSpace.diff and assert.diff. I have started a full regression test suite run and will post the results as soon as it has finished.
The patch makes the following changes:
1) Makes maxFieldSize private and fixes its comment
2) Asserts that slotFieldSize/slotEntrySize have been properly initialized in initSpace()
3) Delays the call to initSpace() until slotFieldSize/slotEntrySize have been initialized
> Possible bug in interaction with buffer manager causing pages not to be freed on rollback to savepoint
> ------------------------------------------------------------------------------------------------------
>
> Key: DERBY-3099
> URL: https://issues.apache.org/jira/browse/DERBY-3099
> Project: Derby
> Issue Type: Bug
> Components: Services, Store
> Affects Versions: 10.4.0.0
> Reporter: Knut Anders Hatlen
> Assignee: Knut Anders Hatlen
> Attachments: assert.diff, d3099-1.diff, failfast.diff, initSpace.diff
>
>
> I noticed this strange behaviour when I was working on DERBY-2911. It seems like the result of test case P042 in unit/T_RawStoreFactory.unit is dependent on the actual contents of the buffer manager (which pages have been evicted, which free entries have been reused and so on). I'm not sure if this is a bug in the test or somewhere in the code, or if this is expected behaviour, but it sounds a bit suspicious.
> For instance, commenting out all the test cases preceeding P042 makes P042 fail, even though P042 creates a new container so that it should not be affected by any of the previous test cases. Also, commenting out a space optimization in Clock.findFreeItem() so that freed entries are not reused except when rotateClock() is called on a full cache to find a victim, causes the test case to fail. A third way to make it fail, is to vary the scan direction when looking for a free entry to reuse in the new buffer manager (ConcurrentCache). If the scan is disabled or walks the clock from the beginning to the end, the test fails, but if the clock is scanned backwards, it passes.
> The code that fails in the test, is
> c = t_util.t_openContainer(t, segment, cid, true);
> Page checkNextPage = t_util.t_addPage(c);
> if (checkNextPage.getPageNumber() == nextPageNumber)
> throw T_Fail.testFailMsg(
> "expect some pages to be freed by update rollback");
> The expected page number is 2, and the actual page number is 7.
> Before this, a large row has been inserted on page 1 and overflows to page 2, 3, 4, 5 and 6. Also, page 7 has been added manually before all the updates were rolled back so that the pages from 2 up to 7 should be unallocated. Page 7 is then added and removed, and the transaction is committed. After reopening the container, the test expects the pages from 2 up to 7 to be free, and that t_addPage() should allocate page number 2.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: [jira] Updated: (DERBY-3099) Possible bug in interaction with
buffer manager causing pages not to be freed on rollback to savepoint
Posted by Bryan Pendleton <bp...@amberpoint.com>.
> The attached patch delays the call to initSpace() until
> slotEntrySize is properly initialized. This made T_RawStoreFactory
> run successfully with the buffer manager changes that made it fail before.
Yay! This seems like an important and valuable finding. Thank you
very much for taking the time to really dig into this one.
thanks,
bryan
[jira] Updated: (DERBY-3099) Possible bug in interaction with
buffer manager causing pages not to be freed on rollback to savepoint
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Knut Anders Hatlen updated DERBY-3099:
--------------------------------------
Attachment: initSpace.diff
I think I found the problem. I extended the toString() methods in the page classes so that they printed all the fields, and then I noticed that the maxFieldSize field was different too. This field is initialized in StoredPage.initSpace() and depends on the value of slotEntrySize. initSpace() is called from StoredPage.usePageBuffer() which also initializes slotEntrySize. However, slotEntrySize is not initialized until after initSpace() has been called, so the value seen in initSpace() is old and possibly incorrect.
The attached patch delays the call to initSpace() until slotEntrySize is properly initialized. This made T_RawStoreFactory run successfully with the buffer manager changes that made it fail before.
> Possible bug in interaction with buffer manager causing pages not to be freed on rollback to savepoint
> ------------------------------------------------------------------------------------------------------
>
> Key: DERBY-3099
> URL: https://issues.apache.org/jira/browse/DERBY-3099
> Project: Derby
> Issue Type: Bug
> Components: Services, Store
> Affects Versions: 10.4.0.0
> Reporter: Knut Anders Hatlen
> Attachments: failfast.diff, initSpace.diff
>
>
> I noticed this strange behaviour when I was working on DERBY-2911. It seems like the result of test case P042 in unit/T_RawStoreFactory.unit is dependent on the actual contents of the buffer manager (which pages have been evicted, which free entries have been reused and so on). I'm not sure if this is a bug in the test or somewhere in the code, or if this is expected behaviour, but it sounds a bit suspicious.
> For instance, commenting out all the test cases preceeding P042 makes P042 fail, even though P042 creates a new container so that it should not be affected by any of the previous test cases. Also, commenting out a space optimization in Clock.findFreeItem() so that freed entries are not reused except when rotateClock() is called on a full cache to find a victim, causes the test case to fail. A third way to make it fail, is to vary the scan direction when looking for a free entry to reuse in the new buffer manager (ConcurrentCache). If the scan is disabled or walks the clock from the beginning to the end, the test fails, but if the clock is scanned backwards, it passes.
> The code that fails in the test, is
> c = t_util.t_openContainer(t, segment, cid, true);
> Page checkNextPage = t_util.t_addPage(c);
> if (checkNextPage.getPageNumber() == nextPageNumber)
> throw T_Fail.testFailMsg(
> "expect some pages to be freed by update rollback");
> The expected page number is 2, and the actual page number is 7.
> Before this, a large row has been inserted on page 1 and overflows to page 2, 3, 4, 5 and 6. Also, page 7 has been added manually before all the updates were rolled back so that the pages from 2 up to 7 should be unallocated. Page 7 is then added and removed, and the transaction is committed. After reopening the container, the test expects the pages from 2 up to 7 to be free, and that t_addPage() should allocate page number 2.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-3099) Possible bug in interaction with
buffer manager causing pages not to be freed on rollback to savepoint
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12532053 ]
Knut Anders Hatlen commented on DERBY-3099:
-------------------------------------------
Here is a dump of the alloc page right before the call to addPage() in a successfull test run (clean trunk). Note that page 3, 4, 5 and 6 are marked as valid, in use pages. I had expected those pages to be free, since the update that overflowed to them was rolled back.
*** Alloc page ***
nextAllocPageNumber = -1
nextAllocPageOffset = 0
reserved1 = 0
reserved2 = 0
reserved3 = 0
reserved4 = 0
borrowedSpace = 80
extent = -----------------------------------------------------------------------
-------
Extent map of from page 1 to page 10216
page 1: valid, in use page
page 2: free page
page 3: valid, in use page
page 4: valid, in use page
page 5: valid, in use page
page 6: valid, in use page
page 7: free page
From 7 to 10216 are un-allocated pages
------------------------------------------------------------------------------
---------------------------------------------------
page id Page(0,Container(0, 1191405214574)) Overflow: false PageVersion: 28 SlotsInUse: 0 DeletedRowCount: 0 PageStatus: 1 NextId: 6 firstFreeByte: 189 freeSpace: 3899 totalSpace: 3899 spareSpace: 0 PageSize: 4096
---------------------------------------------------
Hex dump:
00000000: 0076 0000 0001 0000 0000 0000 0000 0000 .v..............
00000010: 0000 0006 0000 0000 0000 0000 0000 0000 ................
00000020: 0000 0000 0001 0000 0000 0000 0000 0000 ................
00000030: 0000 0000 0000 0000 0000 0000 ffff ffff ................
00000040: ffff ffff 0000 0000 0000 0000 0000 0000 ................
00000050: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000060: 0000 0000 0000 0000 0000 0000 5000 0000 ............P...
00000070: 7400 0000 0000 0010 0000 0000 1400 0000 t...............
00000080: 0c00 0100 0800 0000 0000 0000 0000 0000 ................
00000090: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000000a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000000b0: 0000 0000 0000 0000 00dc 7f30 ca00 0000 .........?.0?...
000000c0: 0000 0010 0000 0000 0000 0000 0100 0000 ................
000000d0: 0000 0027 e800 0000 0030 0000 0000 0000 .........0......
000000e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
.
.
.
(the rest of the page is just zeros, except the checksum)
> Possible bug in interaction with buffer manager causing pages not to be freed on rollback to savepoint
> ------------------------------------------------------------------------------------------------------
>
> Key: DERBY-3099
> URL: https://issues.apache.org/jira/browse/DERBY-3099
> Project: Derby
> Issue Type: Bug
> Components: Services, Store
> Affects Versions: 10.4.0.0
> Reporter: Knut Anders Hatlen
> Attachments: failfast.diff
>
>
> I noticed this strange behaviour when I was working on DERBY-2911. It seems like the result of test case P042 in unit/T_RawStoreFactory.unit is dependent on the actual contents of the buffer manager (which pages have been evicted, which free entries have been reused and so on). I'm not sure if this is a bug in the test or somewhere in the code, or if this is expected behaviour, but it sounds a bit suspicious.
> For instance, commenting out all the test cases preceeding P042 makes P042 fail, even though P042 creates a new container so that it should not be affected by any of the previous test cases. Also, commenting out a space optimization in Clock.findFreeItem() so that freed entries are not reused except when rotateClock() is called on a full cache to find a victim, causes the test case to fail. A third way to make it fail, is to vary the scan direction when looking for a free entry to reuse in the new buffer manager (ConcurrentCache). If the scan is disabled or walks the clock from the beginning to the end, the test fails, but if the clock is scanned backwards, it passes.
> The code that fails in the test, is
> c = t_util.t_openContainer(t, segment, cid, true);
> Page checkNextPage = t_util.t_addPage(c);
> if (checkNextPage.getPageNumber() == nextPageNumber)
> throw T_Fail.testFailMsg(
> "expect some pages to be freed by update rollback");
> The expected page number is 2, and the actual page number is 7.
> Before this, a large row has been inserted on page 1 and overflows to page 2, 3, 4, 5 and 6. Also, page 7 has been added manually before all the updates were rolled back so that the pages from 2 up to 7 should be unallocated. Page 7 is then added and removed, and the transaction is committed. After reopening the container, the test expects the pages from 2 up to 7 to be free, and that t_addPage() should allocate page number 2.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-3099) Possible bug in interaction with
buffer manager causing pages not to be freed on rollback to savepoint
Posted by "Bryan Pendleton (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12533766 ]
Bryan Pendleton commented on DERBY-3099:
----------------------------------------
Patch d3099-1.diff seems like a good patch to me. My only suggestion is
that I think you could place a comment into the code around line 700,
noting that the order of operations here is important, something along
the lines of:
// Note that the slotFieldSize and slotEntrySize need to be
// calculated BEFORE initSpace() is called, because the
// maxFieldSize computation in initSpace() includes these
// values in its calculations.
The comment at line 3355 is good, too, but it is somewhat
discontiguous from the order-critical portion of your change.
Whether or not you think an extra comment would help, +1 to commit from me.
> Possible bug in interaction with buffer manager causing pages not to be freed on rollback to savepoint
> ------------------------------------------------------------------------------------------------------
>
> Key: DERBY-3099
> URL: https://issues.apache.org/jira/browse/DERBY-3099
> Project: Derby
> Issue Type: Bug
> Components: Services, Store
> Affects Versions: 10.4.0.0
> Reporter: Knut Anders Hatlen
> Assignee: Knut Anders Hatlen
> Attachments: assert.diff, d3099-1.diff, failfast.diff, initSpace.diff
>
>
> I noticed this strange behaviour when I was working on DERBY-2911. It seems like the result of test case P042 in unit/T_RawStoreFactory.unit is dependent on the actual contents of the buffer manager (which pages have been evicted, which free entries have been reused and so on). I'm not sure if this is a bug in the test or somewhere in the code, or if this is expected behaviour, but it sounds a bit suspicious.
> For instance, commenting out all the test cases preceeding P042 makes P042 fail, even though P042 creates a new container so that it should not be affected by any of the previous test cases. Also, commenting out a space optimization in Clock.findFreeItem() so that freed entries are not reused except when rotateClock() is called on a full cache to find a victim, causes the test case to fail. A third way to make it fail, is to vary the scan direction when looking for a free entry to reuse in the new buffer manager (ConcurrentCache). If the scan is disabled or walks the clock from the beginning to the end, the test fails, but if the clock is scanned backwards, it passes.
> The code that fails in the test, is
> c = t_util.t_openContainer(t, segment, cid, true);
> Page checkNextPage = t_util.t_addPage(c);
> if (checkNextPage.getPageNumber() == nextPageNumber)
> throw T_Fail.testFailMsg(
> "expect some pages to be freed by update rollback");
> The expected page number is 2, and the actual page number is 7.
> Before this, a large row has been inserted on page 1 and overflows to page 2, 3, 4, 5 and 6. Also, page 7 has been added manually before all the updates were rolled back so that the pages from 2 up to 7 should be unallocated. Page 7 is then added and removed, and the transaction is committed. After reopening the container, the test expects the pages from 2 up to 7 to be free, and that t_addPage() should allocate page number 2.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-3099) Possible bug in interaction with
buffer manager causing pages not to be freed on rollback to savepoint
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12532096 ]
Knut Anders Hatlen commented on DERBY-3099:
-------------------------------------------
To disable the scan for free entries in a buffer manager which is not full, make this change to Clock.findFreeItem():
@@ -860,7 +860,7 @@
// no need to sync on getting the sizes since if they are
// wrong we will discover it in the loop.
- if (validItemCount < holders.size()) {
+ if (false&&validItemCount < holders.size()) {
synchronized (this) {
Below is a dump of the alloc page at the same point in the test when running with the scan disabled. Major differences:
a) page 2 is not free
b) 6th byte (status byte) is zero, whereas the only valid values are 1 (valid) and 2 (invalid). However, the pageStatus flag is 1.
c) firstFreeByte, freeSpace and totalSpace are different
*** Alloc page ***
nextAllocPageNumber = -1
nextAllocPageOffset = 0
reserved1 = 0
reserved2 = 0
reserved3 = 0
reserved4 = 0
borrowedSpace = 80
extent = ------------------------------------------------------------------------------
Extent map of from page 1 to page 10216
page 1: valid, in use page
page 2: valid, in use page
page 3: valid, in use page
page 4: valid, in use page
page 5: valid, in use page
page 6: valid, in use page
page 7: free page
From 7 to 10216 are un-allocated pages
------------------------------------------------------------------------------
---------------------------------------------------
page id Page(0,Container(0, 1191416310107)) Overflow: false PageVersion: 27 SlotsInUse: 0 DeletedRowCount: 0 PageStatus: 1 NextId: 6 firstFreeByte: 109 freeSpace: 3979 totalSpace: 3979 spareSpace: 0 PageSize: 4096
---------------------------------------------------
Hex dump:
00000000: 0076 0000 0000 0000 0000 0000 0000 0000 .v..............
00000010: 0000 0006 0000 0000 0000 0000 0000 0000 ................
00000020: 0000 0000 0001 0000 0000 0000 0000 0000 ................
00000030: 0000 0000 0000 0000 0000 0000 ffff ffff ................
00000040: ffff ffff 0000 0000 0000 0000 0000 0000 ................
00000050: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000060: 0000 0000 0000 0000 0000 0000 5000 0000 ............P...
00000070: 7400 0000 0000 0010 0000 0000 1400 0000 t...............
00000080: 0c00 0100 0800 0000 0000 0000 0000 0000 ................
00000090: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000000a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000000b0: 0000 0000 0000 0000 00dc 7f30 ca00 0000 .........?.0?...
000000c0: 0000 0010 0000 0000 0000 0000 0100 0000 ................
000000d0: 0000 0027 e800 0000 0030 0000 0000 0000 .........0......
000000e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
.
.
.
> Possible bug in interaction with buffer manager causing pages not to be freed on rollback to savepoint
> ------------------------------------------------------------------------------------------------------
>
> Key: DERBY-3099
> URL: https://issues.apache.org/jira/browse/DERBY-3099
> Project: Derby
> Issue Type: Bug
> Components: Services, Store
> Affects Versions: 10.4.0.0
> Reporter: Knut Anders Hatlen
> Attachments: failfast.diff
>
>
> I noticed this strange behaviour when I was working on DERBY-2911. It seems like the result of test case P042 in unit/T_RawStoreFactory.unit is dependent on the actual contents of the buffer manager (which pages have been evicted, which free entries have been reused and so on). I'm not sure if this is a bug in the test or somewhere in the code, or if this is expected behaviour, but it sounds a bit suspicious.
> For instance, commenting out all the test cases preceeding P042 makes P042 fail, even though P042 creates a new container so that it should not be affected by any of the previous test cases. Also, commenting out a space optimization in Clock.findFreeItem() so that freed entries are not reused except when rotateClock() is called on a full cache to find a victim, causes the test case to fail. A third way to make it fail, is to vary the scan direction when looking for a free entry to reuse in the new buffer manager (ConcurrentCache). If the scan is disabled or walks the clock from the beginning to the end, the test fails, but if the clock is scanned backwards, it passes.
> The code that fails in the test, is
> c = t_util.t_openContainer(t, segment, cid, true);
> Page checkNextPage = t_util.t_addPage(c);
> if (checkNextPage.getPageNumber() == nextPageNumber)
> throw T_Fail.testFailMsg(
> "expect some pages to be freed by update rollback");
> The expected page number is 2, and the actual page number is 7.
> Before this, a large row has been inserted on page 1 and overflows to page 2, 3, 4, 5 and 6. Also, page 7 has been added manually before all the updates were rolled back so that the pages from 2 up to 7 should be unallocated. Page 7 is then added and removed, and the transaction is committed. After reopening the container, the test expects the pages from 2 up to 7 to be free, and that t_addPage() should allocate page number 2.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-3099) Possible bug in interaction with
buffer manager causing pages not to be freed on rollback to savepoint
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12532106 ]
Knut Anders Hatlen commented on DERBY-3099:
-------------------------------------------
If I run the test with all other test cases commented out (but with a clean version of Clock), the status byte is OK (1), but all the other differences are there (page 2 is not free, and firstFreeByte/freeSpace/totalSpace are different).
So I guess the first thing we need to find out is why firstFreeByte/freeSpace/totalSpace are different already when the container is created.
> Possible bug in interaction with buffer manager causing pages not to be freed on rollback to savepoint
> ------------------------------------------------------------------------------------------------------
>
> Key: DERBY-3099
> URL: https://issues.apache.org/jira/browse/DERBY-3099
> Project: Derby
> Issue Type: Bug
> Components: Services, Store
> Affects Versions: 10.4.0.0
> Reporter: Knut Anders Hatlen
> Attachments: failfast.diff
>
>
> I noticed this strange behaviour when I was working on DERBY-2911. It seems like the result of test case P042 in unit/T_RawStoreFactory.unit is dependent on the actual contents of the buffer manager (which pages have been evicted, which free entries have been reused and so on). I'm not sure if this is a bug in the test or somewhere in the code, or if this is expected behaviour, but it sounds a bit suspicious.
> For instance, commenting out all the test cases preceeding P042 makes P042 fail, even though P042 creates a new container so that it should not be affected by any of the previous test cases. Also, commenting out a space optimization in Clock.findFreeItem() so that freed entries are not reused except when rotateClock() is called on a full cache to find a victim, causes the test case to fail. A third way to make it fail, is to vary the scan direction when looking for a free entry to reuse in the new buffer manager (ConcurrentCache). If the scan is disabled or walks the clock from the beginning to the end, the test fails, but if the clock is scanned backwards, it passes.
> The code that fails in the test, is
> c = t_util.t_openContainer(t, segment, cid, true);
> Page checkNextPage = t_util.t_addPage(c);
> if (checkNextPage.getPageNumber() == nextPageNumber)
> throw T_Fail.testFailMsg(
> "expect some pages to be freed by update rollback");
> The expected page number is 2, and the actual page number is 7.
> Before this, a large row has been inserted on page 1 and overflows to page 2, 3, 4, 5 and 6. Also, page 7 has been added manually before all the updates were rolled back so that the pages from 2 up to 7 should be unallocated. Page 7 is then added and removed, and the transaction is committed. After reopening the container, the test expects the pages from 2 up to 7 to be free, and that t_addPage() should allocate page number 2.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-3099) Possible bug in interaction with
buffer manager causing pages not to be freed on rollback to savepoint
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12533135 ]
Knut Anders Hatlen commented on DERBY-3099:
-------------------------------------------
OK, so I found out why firstFreeByte/freeSpace/totalSpace were different (see DERBY-3116), but that doesn't seem to be the problem. I'll keep looking.
> Possible bug in interaction with buffer manager causing pages not to be freed on rollback to savepoint
> ------------------------------------------------------------------------------------------------------
>
> Key: DERBY-3099
> URL: https://issues.apache.org/jira/browse/DERBY-3099
> Project: Derby
> Issue Type: Bug
> Components: Services, Store
> Affects Versions: 10.4.0.0
> Reporter: Knut Anders Hatlen
> Attachments: failfast.diff
>
>
> I noticed this strange behaviour when I was working on DERBY-2911. It seems like the result of test case P042 in unit/T_RawStoreFactory.unit is dependent on the actual contents of the buffer manager (which pages have been evicted, which free entries have been reused and so on). I'm not sure if this is a bug in the test or somewhere in the code, or if this is expected behaviour, but it sounds a bit suspicious.
> For instance, commenting out all the test cases preceeding P042 makes P042 fail, even though P042 creates a new container so that it should not be affected by any of the previous test cases. Also, commenting out a space optimization in Clock.findFreeItem() so that freed entries are not reused except when rotateClock() is called on a full cache to find a victim, causes the test case to fail. A third way to make it fail, is to vary the scan direction when looking for a free entry to reuse in the new buffer manager (ConcurrentCache). If the scan is disabled or walks the clock from the beginning to the end, the test fails, but if the clock is scanned backwards, it passes.
> The code that fails in the test, is
> c = t_util.t_openContainer(t, segment, cid, true);
> Page checkNextPage = t_util.t_addPage(c);
> if (checkNextPage.getPageNumber() == nextPageNumber)
> throw T_Fail.testFailMsg(
> "expect some pages to be freed by update rollback");
> The expected page number is 2, and the actual page number is 7.
> Before this, a large row has been inserted on page 1 and overflows to page 2, 3, 4, 5 and 6. Also, page 7 has been added manually before all the updates were rolled back so that the pages from 2 up to 7 should be unallocated. Page 7 is then added and removed, and the transaction is committed. After reopening the container, the test expects the pages from 2 up to 7 to be free, and that t_addPage() should allocate page number 2.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-3099) Possible bug in interaction with
buffer manager causing pages not to be freed on rollback to savepoint
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Knut Anders Hatlen updated DERBY-3099:
--------------------------------------
Affects Version/s: 10.3.1.4
The bug is also present on the 10.3 branch (I applied the assert.diff patch and got the expected assert failures when I tried to connect to a database). Should the fix be backported, or is it safer just to leave it as it is since there are no known user-visible consequences?
> Possible bug in interaction with buffer manager causing pages not to be freed on rollback to savepoint
> ------------------------------------------------------------------------------------------------------
>
> Key: DERBY-3099
> URL: https://issues.apache.org/jira/browse/DERBY-3099
> Project: Derby
> Issue Type: Bug
> Components: Services, Store
> Affects Versions: 10.3.1.4, 10.4.0.0
> Reporter: Knut Anders Hatlen
> Assignee: Knut Anders Hatlen
> Fix For: 10.4.0.0
>
> Attachments: assert.diff, d3099-1.diff, failfast.diff, initSpace.diff
>
>
> I noticed this strange behaviour when I was working on DERBY-2911. It seems like the result of test case P042 in unit/T_RawStoreFactory.unit is dependent on the actual contents of the buffer manager (which pages have been evicted, which free entries have been reused and so on). I'm not sure if this is a bug in the test or somewhere in the code, or if this is expected behaviour, but it sounds a bit suspicious.
> For instance, commenting out all the test cases preceeding P042 makes P042 fail, even though P042 creates a new container so that it should not be affected by any of the previous test cases. Also, commenting out a space optimization in Clock.findFreeItem() so that freed entries are not reused except when rotateClock() is called on a full cache to find a victim, causes the test case to fail. A third way to make it fail, is to vary the scan direction when looking for a free entry to reuse in the new buffer manager (ConcurrentCache). If the scan is disabled or walks the clock from the beginning to the end, the test fails, but if the clock is scanned backwards, it passes.
> The code that fails in the test, is
> c = t_util.t_openContainer(t, segment, cid, true);
> Page checkNextPage = t_util.t_addPage(c);
> if (checkNextPage.getPageNumber() == nextPageNumber)
> throw T_Fail.testFailMsg(
> "expect some pages to be freed by update rollback");
> The expected page number is 2, and the actual page number is 7.
> Before this, a large row has been inserted on page 1 and overflows to page 2, 3, 4, 5 and 6. Also, page 7 has been added manually before all the updates were rolled back so that the pages from 2 up to 7 should be unallocated. Page 7 is then added and removed, and the transaction is committed. After reopening the container, the test expects the pages from 2 up to 7 to be free, and that t_addPage() should allocate page number 2.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-3099) Possible bug in interaction with
buffer manager causing pages not to be freed on rollback to savepoint
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12532101 ]
Knut Anders Hatlen commented on DERBY-3099:
-------------------------------------------
I also tried to dump the alloc page right after the container was created, and I see the same differences in the 6th byte and in the fields firstFreeByte, freeSpace and totalSpace there as well.
> Possible bug in interaction with buffer manager causing pages not to be freed on rollback to savepoint
> ------------------------------------------------------------------------------------------------------
>
> Key: DERBY-3099
> URL: https://issues.apache.org/jira/browse/DERBY-3099
> Project: Derby
> Issue Type: Bug
> Components: Services, Store
> Affects Versions: 10.4.0.0
> Reporter: Knut Anders Hatlen
> Attachments: failfast.diff
>
>
> I noticed this strange behaviour when I was working on DERBY-2911. It seems like the result of test case P042 in unit/T_RawStoreFactory.unit is dependent on the actual contents of the buffer manager (which pages have been evicted, which free entries have been reused and so on). I'm not sure if this is a bug in the test or somewhere in the code, or if this is expected behaviour, but it sounds a bit suspicious.
> For instance, commenting out all the test cases preceeding P042 makes P042 fail, even though P042 creates a new container so that it should not be affected by any of the previous test cases. Also, commenting out a space optimization in Clock.findFreeItem() so that freed entries are not reused except when rotateClock() is called on a full cache to find a victim, causes the test case to fail. A third way to make it fail, is to vary the scan direction when looking for a free entry to reuse in the new buffer manager (ConcurrentCache). If the scan is disabled or walks the clock from the beginning to the end, the test fails, but if the clock is scanned backwards, it passes.
> The code that fails in the test, is
> c = t_util.t_openContainer(t, segment, cid, true);
> Page checkNextPage = t_util.t_addPage(c);
> if (checkNextPage.getPageNumber() == nextPageNumber)
> throw T_Fail.testFailMsg(
> "expect some pages to be freed by update rollback");
> The expected page number is 2, and the actual page number is 7.
> Before this, a large row has been inserted on page 1 and overflows to page 2, 3, 4, 5 and 6. Also, page 7 has been added manually before all the updates were rolled back so that the pages from 2 up to 7 should be unallocated. Page 7 is then added and removed, and the transaction is committed. After reopening the container, the test expects the pages from 2 up to 7 to be free, and that t_addPage() should allocate page number 2.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-3099) Possible bug in interaction with
buffer manager causing pages not to be freed on rollback to savepoint
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12533840 ]
Knut Anders Hatlen commented on DERBY-3099:
-------------------------------------------
Good idea, Bryan. I'll add the comment before I commit. Thanks!
> Possible bug in interaction with buffer manager causing pages not to be freed on rollback to savepoint
> ------------------------------------------------------------------------------------------------------
>
> Key: DERBY-3099
> URL: https://issues.apache.org/jira/browse/DERBY-3099
> Project: Derby
> Issue Type: Bug
> Components: Services, Store
> Affects Versions: 10.4.0.0
> Reporter: Knut Anders Hatlen
> Assignee: Knut Anders Hatlen
> Attachments: assert.diff, d3099-1.diff, failfast.diff, initSpace.diff
>
>
> I noticed this strange behaviour when I was working on DERBY-2911. It seems like the result of test case P042 in unit/T_RawStoreFactory.unit is dependent on the actual contents of the buffer manager (which pages have been evicted, which free entries have been reused and so on). I'm not sure if this is a bug in the test or somewhere in the code, or if this is expected behaviour, but it sounds a bit suspicious.
> For instance, commenting out all the test cases preceeding P042 makes P042 fail, even though P042 creates a new container so that it should not be affected by any of the previous test cases. Also, commenting out a space optimization in Clock.findFreeItem() so that freed entries are not reused except when rotateClock() is called on a full cache to find a victim, causes the test case to fail. A third way to make it fail, is to vary the scan direction when looking for a free entry to reuse in the new buffer manager (ConcurrentCache). If the scan is disabled or walks the clock from the beginning to the end, the test fails, but if the clock is scanned backwards, it passes.
> The code that fails in the test, is
> c = t_util.t_openContainer(t, segment, cid, true);
> Page checkNextPage = t_util.t_addPage(c);
> if (checkNextPage.getPageNumber() == nextPageNumber)
> throw T_Fail.testFailMsg(
> "expect some pages to be freed by update rollback");
> The expected page number is 2, and the actual page number is 7.
> Before this, a large row has been inserted on page 1 and overflows to page 2, 3, 4, 5 and 6. Also, page 7 has been added manually before all the updates were rolled back so that the pages from 2 up to 7 should be unallocated. Page 7 is then added and removed, and the transaction is committed. After reopening the container, the test expects the pages from 2 up to 7 to be free, and that t_addPage() should allocate page number 2.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Closed: (DERBY-3099) Possible bug in interaction with buffer
manager causing pages not to be freed on rollback to savepoint
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Knut Anders Hatlen closed DERBY-3099.
-------------------------------------
> Possible bug in interaction with buffer manager causing pages not to be freed on rollback to savepoint
> ------------------------------------------------------------------------------------------------------
>
> Key: DERBY-3099
> URL: https://issues.apache.org/jira/browse/DERBY-3099
> Project: Derby
> Issue Type: Bug
> Components: Services, Store
> Affects Versions: 10.3.1.4, 10.4.0.0
> Reporter: Knut Anders Hatlen
> Assignee: Knut Anders Hatlen
> Fix For: 10.3.1.5, 10.4.0.0
>
> Attachments: assert.diff, d3099-1.diff, failfast.diff, initSpace.diff
>
>
> I noticed this strange behaviour when I was working on DERBY-2911. It seems like the result of test case P042 in unit/T_RawStoreFactory.unit is dependent on the actual contents of the buffer manager (which pages have been evicted, which free entries have been reused and so on). I'm not sure if this is a bug in the test or somewhere in the code, or if this is expected behaviour, but it sounds a bit suspicious.
> For instance, commenting out all the test cases preceeding P042 makes P042 fail, even though P042 creates a new container so that it should not be affected by any of the previous test cases. Also, commenting out a space optimization in Clock.findFreeItem() so that freed entries are not reused except when rotateClock() is called on a full cache to find a victim, causes the test case to fail. A third way to make it fail, is to vary the scan direction when looking for a free entry to reuse in the new buffer manager (ConcurrentCache). If the scan is disabled or walks the clock from the beginning to the end, the test fails, but if the clock is scanned backwards, it passes.
> The code that fails in the test, is
> c = t_util.t_openContainer(t, segment, cid, true);
> Page checkNextPage = t_util.t_addPage(c);
> if (checkNextPage.getPageNumber() == nextPageNumber)
> throw T_Fail.testFailMsg(
> "expect some pages to be freed by update rollback");
> The expected page number is 2, and the actual page number is 7.
> Before this, a large row has been inserted on page 1 and overflows to page 2, 3, 4, 5 and 6. Also, page 7 has been added manually before all the updates were rolled back so that the pages from 2 up to 7 should be unallocated. Page 7 is then added and removed, and the transaction is committed. After reopening the container, the test expects the pages from 2 up to 7 to be free, and that t_addPage() should allocate page number 2.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-3099) Possible bug in interaction with
buffer manager causing pages not to be freed on rollback to savepoint
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Knut Anders Hatlen updated DERBY-3099:
--------------------------------------
Attachment: failfast.diff
When T_RawStoreFactory.P042() fails, a finally clause tries to unlatch an already unlatched page and hides the actual error. The failure in the finally clause is not immediately picked up by the test framework and the test therefore hangs until it times out after one hour.
The attached patch makes the cleanup code check whether the page is latched before unlatching it. This makes the test fail faster if it fails, and the real error is not hidden by the error in the cleanup code. The patch also removes an unused variable from the test case.
Committed revision 581544.
> Possible bug in interaction with buffer manager causing pages not to be freed on rollback to savepoint
> ------------------------------------------------------------------------------------------------------
>
> Key: DERBY-3099
> URL: https://issues.apache.org/jira/browse/DERBY-3099
> Project: Derby
> Issue Type: Bug
> Components: Services, Store
> Affects Versions: 10.4.0.0
> Reporter: Knut Anders Hatlen
> Attachments: failfast.diff
>
>
> I noticed this strange behaviour when I was working on DERBY-2911. It seems like the result of test case P042 in unit/T_RawStoreFactory.unit is dependent on the actual contents of the buffer manager (which pages have been evicted, which free entries have been reused and so on). I'm not sure if this is a bug in the test or somewhere in the code, or if this is expected behaviour, but it sounds a bit suspicious.
> For instance, commenting out all the test cases preceeding P042 makes P042 fail, even though P042 creates a new container so that it should not be affected by any of the previous test cases. Also, commenting out a space optimization in Clock.findFreeItem() so that freed entries are not reused except when rotateClock() is called on a full cache to find a victim, causes the test case to fail. A third way to make it fail, is to vary the scan direction when looking for a free entry to reuse in the new buffer manager (ConcurrentCache). If the scan is disabled or walks the clock from the beginning to the end, the test fails, but if the clock is scanned backwards, it passes.
> The code that fails in the test, is
> c = t_util.t_openContainer(t, segment, cid, true);
> Page checkNextPage = t_util.t_addPage(c);
> if (checkNextPage.getPageNumber() == nextPageNumber)
> throw T_Fail.testFailMsg(
> "expect some pages to be freed by update rollback");
> The expected page number is 2, and the actual page number is 7.
> Before this, a large row has been inserted on page 1 and overflows to page 2, 3, 4, 5 and 6. Also, page 7 has been added manually before all the updates were rolled back so that the pages from 2 up to 7 should be unallocated. Page 7 is then added and removed, and the transaction is committed. After reopening the container, the test expects the pages from 2 up to 7 to be free, and that t_addPage() should allocate page number 2.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (DERBY-3099) Possible bug in interaction with
buffer manager causing pages not to be freed on rollback to savepoint
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Knut Anders Hatlen resolved DERBY-3099.
---------------------------------------
Resolution: Fixed
Fix Version/s: 10.4.0.0
Derby Info: (was: [Patch Available])
Committed revision 583691.
> Possible bug in interaction with buffer manager causing pages not to be freed on rollback to savepoint
> ------------------------------------------------------------------------------------------------------
>
> Key: DERBY-3099
> URL: https://issues.apache.org/jira/browse/DERBY-3099
> Project: Derby
> Issue Type: Bug
> Components: Services, Store
> Affects Versions: 10.4.0.0
> Reporter: Knut Anders Hatlen
> Assignee: Knut Anders Hatlen
> Fix For: 10.4.0.0
>
> Attachments: assert.diff, d3099-1.diff, failfast.diff, initSpace.diff
>
>
> I noticed this strange behaviour when I was working on DERBY-2911. It seems like the result of test case P042 in unit/T_RawStoreFactory.unit is dependent on the actual contents of the buffer manager (which pages have been evicted, which free entries have been reused and so on). I'm not sure if this is a bug in the test or somewhere in the code, or if this is expected behaviour, but it sounds a bit suspicious.
> For instance, commenting out all the test cases preceeding P042 makes P042 fail, even though P042 creates a new container so that it should not be affected by any of the previous test cases. Also, commenting out a space optimization in Clock.findFreeItem() so that freed entries are not reused except when rotateClock() is called on a full cache to find a victim, causes the test case to fail. A third way to make it fail, is to vary the scan direction when looking for a free entry to reuse in the new buffer manager (ConcurrentCache). If the scan is disabled or walks the clock from the beginning to the end, the test fails, but if the clock is scanned backwards, it passes.
> The code that fails in the test, is
> c = t_util.t_openContainer(t, segment, cid, true);
> Page checkNextPage = t_util.t_addPage(c);
> if (checkNextPage.getPageNumber() == nextPageNumber)
> throw T_Fail.testFailMsg(
> "expect some pages to be freed by update rollback");
> The expected page number is 2, and the actual page number is 7.
> Before this, a large row has been inserted on page 1 and overflows to page 2, 3, 4, 5 and 6. Also, page 7 has been added manually before all the updates were rolled back so that the pages from 2 up to 7 should be unallocated. Page 7 is then added and removed, and the transaction is committed. After reopening the container, the test expects the pages from 2 up to 7 to be free, and that t_addPage() should allocate page number 2.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-3099) Possible bug in interaction with
buffer manager causing pages not to be freed on rollback to savepoint
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Knut Anders Hatlen updated DERBY-3099:
--------------------------------------
Attachment: assert.diff
I don't know how to make a reliable repro for this issue on a clean trunk, but the attached assert.diff adds some asserts that check if slotFieldSize and slotEntrySize are uninitialized. With these asserts, I'm not able to create a new database:
ij version 10.4
ij> connect 'jdbc:derby:db;create=true';
ERROR XJ041: Failed to create database 'db', see the next exception for details.
ERROR XBM01: Startup failed due to an exception. See next exception for details.
ERROR XJ001: Java exception: 'ASSERT FAILED slotFieldSize uninitialized: org.apache.derby.shared.common.sanity.AssertFailure'.
> Possible bug in interaction with buffer manager causing pages not to be freed on rollback to savepoint
> ------------------------------------------------------------------------------------------------------
>
> Key: DERBY-3099
> URL: https://issues.apache.org/jira/browse/DERBY-3099
> Project: Derby
> Issue Type: Bug
> Components: Services, Store
> Affects Versions: 10.4.0.0
> Reporter: Knut Anders Hatlen
> Assignee: Knut Anders Hatlen
> Attachments: assert.diff, failfast.diff, initSpace.diff
>
>
> I noticed this strange behaviour when I was working on DERBY-2911. It seems like the result of test case P042 in unit/T_RawStoreFactory.unit is dependent on the actual contents of the buffer manager (which pages have been evicted, which free entries have been reused and so on). I'm not sure if this is a bug in the test or somewhere in the code, or if this is expected behaviour, but it sounds a bit suspicious.
> For instance, commenting out all the test cases preceeding P042 makes P042 fail, even though P042 creates a new container so that it should not be affected by any of the previous test cases. Also, commenting out a space optimization in Clock.findFreeItem() so that freed entries are not reused except when rotateClock() is called on a full cache to find a victim, causes the test case to fail. A third way to make it fail, is to vary the scan direction when looking for a free entry to reuse in the new buffer manager (ConcurrentCache). If the scan is disabled or walks the clock from the beginning to the end, the test fails, but if the clock is scanned backwards, it passes.
> The code that fails in the test, is
> c = t_util.t_openContainer(t, segment, cid, true);
> Page checkNextPage = t_util.t_addPage(c);
> if (checkNextPage.getPageNumber() == nextPageNumber)
> throw T_Fail.testFailMsg(
> "expect some pages to be freed by update rollback");
> The expected page number is 2, and the actual page number is 7.
> Before this, a large row has been inserted on page 1 and overflows to page 2, 3, 4, 5 and 6. Also, page 7 has been added manually before all the updates were rolled back so that the pages from 2 up to 7 should be unallocated. Page 7 is then added and removed, and the transaction is committed. After reopening the container, the test expects the pages from 2 up to 7 to be free, and that t_addPage() should allocate page number 2.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (DERBY-3099) Possible bug in interaction with
buffer manager causing pages not to be freed on rollback to savepoint
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Knut Anders Hatlen reassigned DERBY-3099:
-----------------------------------------
Assignee: Knut Anders Hatlen
> Possible bug in interaction with buffer manager causing pages not to be freed on rollback to savepoint
> ------------------------------------------------------------------------------------------------------
>
> Key: DERBY-3099
> URL: https://issues.apache.org/jira/browse/DERBY-3099
> Project: Derby
> Issue Type: Bug
> Components: Services, Store
> Affects Versions: 10.4.0.0
> Reporter: Knut Anders Hatlen
> Assignee: Knut Anders Hatlen
> Attachments: failfast.diff, initSpace.diff
>
>
> I noticed this strange behaviour when I was working on DERBY-2911. It seems like the result of test case P042 in unit/T_RawStoreFactory.unit is dependent on the actual contents of the buffer manager (which pages have been evicted, which free entries have been reused and so on). I'm not sure if this is a bug in the test or somewhere in the code, or if this is expected behaviour, but it sounds a bit suspicious.
> For instance, commenting out all the test cases preceeding P042 makes P042 fail, even though P042 creates a new container so that it should not be affected by any of the previous test cases. Also, commenting out a space optimization in Clock.findFreeItem() so that freed entries are not reused except when rotateClock() is called on a full cache to find a victim, causes the test case to fail. A third way to make it fail, is to vary the scan direction when looking for a free entry to reuse in the new buffer manager (ConcurrentCache). If the scan is disabled or walks the clock from the beginning to the end, the test fails, but if the clock is scanned backwards, it passes.
> The code that fails in the test, is
> c = t_util.t_openContainer(t, segment, cid, true);
> Page checkNextPage = t_util.t_addPage(c);
> if (checkNextPage.getPageNumber() == nextPageNumber)
> throw T_Fail.testFailMsg(
> "expect some pages to be freed by update rollback");
> The expected page number is 2, and the actual page number is 7.
> Before this, a large row has been inserted on page 1 and overflows to page 2, 3, 4, 5 and 6. Also, page 7 has been added manually before all the updates were rolled back so that the pages from 2 up to 7 should be unallocated. Page 7 is then added and removed, and the transaction is committed. After reopening the container, the test expects the pages from 2 up to 7 to be free, and that t_addPage() should allocate page number 2.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-3099) Possible bug in interaction with
buffer manager causing pages not to be freed on rollback to savepoint
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Knut Anders Hatlen updated DERBY-3099:
--------------------------------------
Derby Info: [Patch Available]
Derbyall and suites.All ran cleanly. Please review.
> Possible bug in interaction with buffer manager causing pages not to be freed on rollback to savepoint
> ------------------------------------------------------------------------------------------------------
>
> Key: DERBY-3099
> URL: https://issues.apache.org/jira/browse/DERBY-3099
> Project: Derby
> Issue Type: Bug
> Components: Services, Store
> Affects Versions: 10.4.0.0
> Reporter: Knut Anders Hatlen
> Assignee: Knut Anders Hatlen
> Attachments: assert.diff, d3099-1.diff, failfast.diff, initSpace.diff
>
>
> I noticed this strange behaviour when I was working on DERBY-2911. It seems like the result of test case P042 in unit/T_RawStoreFactory.unit is dependent on the actual contents of the buffer manager (which pages have been evicted, which free entries have been reused and so on). I'm not sure if this is a bug in the test or somewhere in the code, or if this is expected behaviour, but it sounds a bit suspicious.
> For instance, commenting out all the test cases preceeding P042 makes P042 fail, even though P042 creates a new container so that it should not be affected by any of the previous test cases. Also, commenting out a space optimization in Clock.findFreeItem() so that freed entries are not reused except when rotateClock() is called on a full cache to find a victim, causes the test case to fail. A third way to make it fail, is to vary the scan direction when looking for a free entry to reuse in the new buffer manager (ConcurrentCache). If the scan is disabled or walks the clock from the beginning to the end, the test fails, but if the clock is scanned backwards, it passes.
> The code that fails in the test, is
> c = t_util.t_openContainer(t, segment, cid, true);
> Page checkNextPage = t_util.t_addPage(c);
> if (checkNextPage.getPageNumber() == nextPageNumber)
> throw T_Fail.testFailMsg(
> "expect some pages to be freed by update rollback");
> The expected page number is 2, and the actual page number is 7.
> Before this, a large row has been inserted on page 1 and overflows to page 2, 3, 4, 5 and 6. Also, page 7 has been added manually before all the updates were rolled back so that the pages from 2 up to 7 should be unallocated. Page 7 is then added and removed, and the transaction is committed. After reopening the container, the test expects the pages from 2 up to 7 to be free, and that t_addPage() should allocate page number 2.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.