You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@asterixdb.apache.org by "Michael Blow (Code Review)" <do...@asterixdb.incubator.apache.org> on 2016/11/07 03:56:55 UTC
Change in asterixdb[master]: ASTERIXDB-1570: "Unable to find free page in buffer cache af...
Michael Blow has uploaded a new change for review.
https://asterix-gerrit.ics.uci.edu/1346
Change subject: ASTERIXDB-1570: "Unable to find free page in buffer cache after 1000 cycles"
......................................................................
ASTERIXDB-1570: "Unable to find free page in buffer cache after 1000 cycles"
Increase test buffer cache size to reliably accomodate 20M values
Change-Id: I620de9ade875f7f7de26aa7495d9fd873824ac83
---
M asterixdb/asterix-app/src/main/resources/asterix-build-configuration.xml
M asterixdb/asterix-common/src/test/java/org/apache/asterix/test/aql/TestExecutor.java
M hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/buffercache/BufferCache.java
3 files changed, 11 insertions(+), 11 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/46/1346/1
diff --git a/asterixdb/asterix-app/src/main/resources/asterix-build-configuration.xml b/asterixdb/asterix-app/src/main/resources/asterix-build-configuration.xml
index ef0cdf2..90dbe5f 100644
--- a/asterixdb/asterix-app/src/main/resources/asterix-build-configuration.xml
+++ b/asterixdb/asterix-app/src/main/resources/asterix-build-configuration.xml
@@ -74,15 +74,15 @@
<name>storage.buffercache.pagesize</name>
<value>32KB</value>
<description>The page size in bytes for pages in the buffer cache.
- (Default = "32768" // 32KB)
+ (Default = "128KB")
</description>
</property>
<property>
<name>storage.buffercache.size</name>
- <value>32MB</value>
+ <value>48MB</value>
<description>The size of memory allocated to the disk buffer cache.
- The value should be a multiple of the buffer cache page size(Default
- = "33554432" // 32MB)
+ The value should be a multiple of the buffer cache page size
+ (Default = "512MB")
</description>
</property>
<property>
diff --git a/asterixdb/asterix-common/src/test/java/org/apache/asterix/test/aql/TestExecutor.java b/asterixdb/asterix-common/src/test/java/org/apache/asterix/test/aql/TestExecutor.java
index c6da1c0..4080325 100644
--- a/asterixdb/asterix-common/src/test/java/org/apache/asterix/test/aql/TestExecutor.java
+++ b/asterixdb/asterix-common/src/test/java/org/apache/asterix/test/aql/TestExecutor.java
@@ -185,8 +185,7 @@
}
lineActual = readerActual.readLine();
if (lineActual != null) {
- throw new ComparisonException(
- "Result for " + scriptFile + " changed at line " + num + ":\n< \n> " + lineActual);
+ throwLineChanged(scriptFile, "<EOF>", lineActual, num);
}
} catch (Exception e) {
System.err.println("Actual results file: " + actualFile.toString());
diff --git a/hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/buffercache/BufferCache.java b/hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/buffercache/BufferCache.java
index ed33cc4..8ff9ab6 100644
--- a/hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/buffercache/BufferCache.java
+++ b/hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/buffercache/BufferCache.java
@@ -1248,7 +1248,7 @@
private ICachedPage getPageLoop(long dpid, int multiplier, boolean confiscate)
throws HyracksDataException {
- final long startingPinCount = masterPinCount.get();
+ final long startingPinCount = DEBUG ? masterPinCount.get() : -1;
int cycleCount = 0;
try {
while (true) {
@@ -1291,15 +1291,16 @@
if (cycleCount > MAX_PIN_ATTEMPT_CYCLES) {
cycleCount = 0; // suppress warning below
throw new HyracksDataException("Unable to find free page in buffer cache after "
- + MAX_PIN_ATTEMPT_CYCLES + " cycles (buffer cache undersized?); "
- + (masterPinCount.get() - startingPinCount) + " successful pins since start of cycle");
+ + MAX_PIN_ATTEMPT_CYCLES + " cycles (buffer cache undersized?)" + (DEBUG ? " ; "
+ + (masterPinCount.get() - startingPinCount) + " successful pins since start of cycle"
+ : ""));
}
}
} finally {
if (cycleCount > PIN_ATTEMPT_CYCLES_WARNING_THRESHOLD && LOGGER.isLoggable(Level.WARNING)) {
LOGGER.warning("Took " + cycleCount + " cycles to find free page in buffer cache. (buffer cache " +
- "undersized?); " + (masterPinCount.get() - startingPinCount) +
- " successful pins since start of cycle");
+ "undersized?)" + (DEBUG ? " ; " + (masterPinCount.get() - startingPinCount) +
+ " successful pins since start of cycle" : ""));
}
}
}
--
To view, visit https://asterix-gerrit.ics.uci.edu/1346
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I620de9ade875f7f7de26aa7495d9fd873824ac83
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Michael Blow <mb...@apache.org>
Change in asterixdb[master]: ASTERIXDB-1570: "Unable to find free page in buffer cache af...
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: ASTERIXDB-1570: "Unable to find free page in buffer cache after 1000 cycles"
......................................................................
Patch Set 2:
Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/3262/
--
To view, visit https://asterix-gerrit.ics.uci.edu/1346
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I620de9ade875f7f7de26aa7495d9fd873824ac83
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Michael Blow <mb...@apache.org>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: ASTERIXDB-1570: "Unable to find free page in buffer cache af...
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: ASTERIXDB-1570: "Unable to find free page in buffer cache after 1000 cycles"
......................................................................
Patch Set 2:
Integration Tests Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1100/
--
To view, visit https://asterix-gerrit.ics.uci.edu/1346
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I620de9ade875f7f7de26aa7495d9fd873824ac83
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Michael Blow <mb...@apache.org>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-HasComments: No
Change in asterixdb[master]: ASTERIXDB-1570: "Unable to find free page in buffer cache af...
Posted by "Michael Blow (Code Review)" <do...@asterixdb.incubator.apache.org>.
Hello Jenkins,
I'd like you to reexamine a change. Please visit
https://asterix-gerrit.ics.uci.edu/1346
to look at the new patch set (#2).
Change subject: ASTERIXDB-1570: "Unable to find free page in buffer cache after 1000 cycles"
......................................................................
ASTERIXDB-1570: "Unable to find free page in buffer cache after 1000 cycles"
Increase test buffer cache size to reliably accomodate 20M values
Change-Id: I620de9ade875f7f7de26aa7495d9fd873824ac83
---
M asterixdb/asterix-app/src/main/resources/asterix-build-configuration.xml
M asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1/cluster_state_1.1.adm
M asterixdb/asterix-common/src/test/java/org/apache/asterix/test/aql/TestExecutor.java
M hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/buffercache/BufferCache.java
4 files changed, 12 insertions(+), 12 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/46/1346/2
--
To view, visit https://asterix-gerrit.ics.uci.edu/1346
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I620de9ade875f7f7de26aa7495d9fd873824ac83
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Michael Blow <mb...@apache.org>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Change in asterixdb[master]: ASTERIXDB-1570: "Unable to find free page in buffer cache af...
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: ASTERIXDB-1570: "Unable to find free page in buffer cache after 1000 cycles"
......................................................................
Patch Set 2:
WARNING: THIS CHANGE CONTAINS CROSS-PRODUCT CHANGES IN:
* asterixdb
* hyracks-fullstack
PLEASE REVIEW CAREFULLY AND LOOK FOR API CHANGES!
--
To view, visit https://asterix-gerrit.ics.uci.edu/1346
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I620de9ade875f7f7de26aa7495d9fd873824ac83
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Michael Blow <mb...@apache.org>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: ASTERIXDB-1570: "Unable to find free page in buffer cache af...
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: ASTERIXDB-1570: "Unable to find free page in buffer cache after 1000 cycles"
......................................................................
Patch Set 1:
Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/3260/
--
To view, visit https://asterix-gerrit.ics.uci.edu/1346
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I620de9ade875f7f7de26aa7495d9fd873824ac83
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Michael Blow <mb...@apache.org>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: ASTERIXDB-1570: "Unable to find free page in buffer cache af...
Posted by "Michael Blow (Code Review)" <do...@asterixdb.incubator.apache.org>.
Michael Blow has submitted this change and it was merged.
Change subject: ASTERIXDB-1570: "Unable to find free page in buffer cache after 1000 cycles"
......................................................................
ASTERIXDB-1570: "Unable to find free page in buffer cache after 1000 cycles"
Increase test buffer cache size to reliably accomodate 20M values
Change-Id: I620de9ade875f7f7de26aa7495d9fd873824ac83
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1346
Sonar-Qube: Jenkins <je...@fulliautomatix.ics.uci.edu>
Tested-by: Jenkins <je...@fulliautomatix.ics.uci.edu>
Integration-Tests: Jenkins <je...@fulliautomatix.ics.uci.edu>
Reviewed-by: Till Westmann <ti...@apache.org>
---
M asterixdb/asterix-app/src/main/resources/asterix-build-configuration.xml
M asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1/cluster_state_1.1.adm
M asterixdb/asterix-common/src/test/java/org/apache/asterix/test/aql/TestExecutor.java
M hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/buffercache/BufferCache.java
4 files changed, 12 insertions(+), 12 deletions(-)
Approvals:
Till Westmann: Looks good to me, approved
Jenkins: Verified; No violations found; Verified
diff --git a/asterixdb/asterix-app/src/main/resources/asterix-build-configuration.xml b/asterixdb/asterix-app/src/main/resources/asterix-build-configuration.xml
index ef0cdf2..90dbe5f 100644
--- a/asterixdb/asterix-app/src/main/resources/asterix-build-configuration.xml
+++ b/asterixdb/asterix-app/src/main/resources/asterix-build-configuration.xml
@@ -74,15 +74,15 @@
<name>storage.buffercache.pagesize</name>
<value>32KB</value>
<description>The page size in bytes for pages in the buffer cache.
- (Default = "32768" // 32KB)
+ (Default = "128KB")
</description>
</property>
<property>
<name>storage.buffercache.size</name>
- <value>32MB</value>
+ <value>48MB</value>
<description>The size of memory allocated to the disk buffer cache.
- The value should be a multiple of the buffer cache page size(Default
- = "33554432" // 32MB)
+ The value should be a multiple of the buffer cache page size
+ (Default = "512MB")
</description>
</property>
<property>
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1/cluster_state_1.1.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1/cluster_state_1.1.adm
index 649f1e8..a20b1a7 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1/cluster_state_1.1.adm
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1/cluster_state_1.1.adm
@@ -55,7 +55,7 @@
"plot.activate": false,
"storage.buffercache.maxopenfiles": 2147483647,
"storage.buffercache.pagesize": 32768,
- "storage.buffercache.size": 33554432,
+ "storage.buffercache.size": 50331648,
"storage.lsm.bloomfilter.falsepositiverate": 0.01,
"storage.memorycomponent.globalbudget": 536870912,
"storage.memorycomponent.numcomponents": 2,
diff --git a/asterixdb/asterix-common/src/test/java/org/apache/asterix/test/aql/TestExecutor.java b/asterixdb/asterix-common/src/test/java/org/apache/asterix/test/aql/TestExecutor.java
index c6da1c0..4080325 100644
--- a/asterixdb/asterix-common/src/test/java/org/apache/asterix/test/aql/TestExecutor.java
+++ b/asterixdb/asterix-common/src/test/java/org/apache/asterix/test/aql/TestExecutor.java
@@ -185,8 +185,7 @@
}
lineActual = readerActual.readLine();
if (lineActual != null) {
- throw new ComparisonException(
- "Result for " + scriptFile + " changed at line " + num + ":\n< \n> " + lineActual);
+ throwLineChanged(scriptFile, "<EOF>", lineActual, num);
}
} catch (Exception e) {
System.err.println("Actual results file: " + actualFile.toString());
diff --git a/hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/buffercache/BufferCache.java b/hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/buffercache/BufferCache.java
index ed33cc4..8ff9ab6 100644
--- a/hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/buffercache/BufferCache.java
+++ b/hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/buffercache/BufferCache.java
@@ -1248,7 +1248,7 @@
private ICachedPage getPageLoop(long dpid, int multiplier, boolean confiscate)
throws HyracksDataException {
- final long startingPinCount = masterPinCount.get();
+ final long startingPinCount = DEBUG ? masterPinCount.get() : -1;
int cycleCount = 0;
try {
while (true) {
@@ -1291,15 +1291,16 @@
if (cycleCount > MAX_PIN_ATTEMPT_CYCLES) {
cycleCount = 0; // suppress warning below
throw new HyracksDataException("Unable to find free page in buffer cache after "
- + MAX_PIN_ATTEMPT_CYCLES + " cycles (buffer cache undersized?); "
- + (masterPinCount.get() - startingPinCount) + " successful pins since start of cycle");
+ + MAX_PIN_ATTEMPT_CYCLES + " cycles (buffer cache undersized?)" + (DEBUG ? " ; "
+ + (masterPinCount.get() - startingPinCount) + " successful pins since start of cycle"
+ : ""));
}
}
} finally {
if (cycleCount > PIN_ATTEMPT_CYCLES_WARNING_THRESHOLD && LOGGER.isLoggable(Level.WARNING)) {
LOGGER.warning("Took " + cycleCount + " cycles to find free page in buffer cache. (buffer cache " +
- "undersized?); " + (masterPinCount.get() - startingPinCount) +
- " successful pins since start of cycle");
+ "undersized?)" + (DEBUG ? " ; " + (masterPinCount.get() - startingPinCount) +
+ " successful pins since start of cycle" : ""));
}
}
}
--
To view, visit https://asterix-gerrit.ics.uci.edu/1346
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I620de9ade875f7f7de26aa7495d9fd873824ac83
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Michael Blow <mb...@apache.org>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Michael Blow <mb...@apache.org>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Change in asterixdb[master]: ASTERIXDB-1570: "Unable to find free page in buffer cache af...
Posted by "Till Westmann (Code Review)" <do...@asterixdb.incubator.apache.org>.
Till Westmann has posted comments on this change.
Change subject: ASTERIXDB-1570: "Unable to find free page in buffer cache after 1000 cycles"
......................................................................
Patch Set 2: Code-Review+2
--
To view, visit https://asterix-gerrit.ics.uci.edu/1346
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I620de9ade875f7f7de26aa7495d9fd873824ac83
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Michael Blow <mb...@apache.org>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-HasComments: No
Change in asterixdb[master]: ASTERIXDB-1570: "Unable to find free page in buffer cache af...
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: ASTERIXDB-1570: "Unable to find free page in buffer cache after 1000 cycles"
......................................................................
Patch Set 2: Integration-Tests+1
Integration Tests Successful
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1100/ : SUCCESS
--
To view, visit https://asterix-gerrit.ics.uci.edu/1346
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I620de9ade875f7f7de26aa7495d9fd873824ac83
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Michael Blow <mb...@apache.org>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-HasComments: No
Change in asterixdb[master]: ASTERIXDB-1570: "Unable to find free page in buffer cache af...
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: ASTERIXDB-1570: "Unable to find free page in buffer cache after 1000 cycles"
......................................................................
Patch Set 1:
WARNING: THIS CHANGE CONTAINS CROSS-PRODUCT CHANGES IN:
* asterixdb
* hyracks-fullstack
PLEASE REVIEW CAREFULLY AND LOOK FOR API CHANGES!
--
To view, visit https://asterix-gerrit.ics.uci.edu/1346
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I620de9ade875f7f7de26aa7495d9fd873824ac83
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Michael Blow <mb...@apache.org>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No