You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@asterixdb.apache.org by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu> on 2021/08/22 00:38:24 UTC
Change in asterixdb[master]: [NO ISSUE][STO] Improve logging
From Ali Alsuliman <al...@gmail.com>:
Ali Alsuliman has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12923 )
Change subject: [NO ISSUE][STO] Improve logging
......................................................................
[NO ISSUE][STO] Improve logging
- user model changes: no
- storage format changes: no
- interface changes: no
Change-Id: I57c05bc089527e745cf1851be0814242ba89ac55
---
M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/PrimaryIndexOperationTracker.java
2 files changed, 14 insertions(+), 7 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/23/12923/1
diff --git a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
index 2fe100e..bd24696 100644
--- a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
+++ b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
@@ -170,9 +170,15 @@
closeIndex(iInfo);
dsInfo.removeIndex(resourceID);
synchronized (dsInfo) {
- if (dsInfo.getReferenceCount() == 0 && dsInfo.isOpen() && dsInfo.getIndexes().isEmpty()
- && !dsInfo.isExternal()) {
+ int referenceCount = dsInfo.getReferenceCount();
+ boolean open = dsInfo.isOpen();
+ boolean empty = dsInfo.getIndexes().isEmpty();
+ if (referenceCount == 0 && open && empty && !dsInfo.isExternal()) {
+ LOGGER.debug("removing dataset {} from cache", dsInfo.getDatasetID());
removeDatasetFromCache(dsInfo.getDatasetID());
+ } else {
+ LOGGER.debug("keeping dataset {} in cache, ref count {}, open {}, indexes count: {}",
+ dsInfo.getDatasetID(), referenceCount, open, dsInfo.getIndexes().size());
}
}
}
@@ -414,9 +420,10 @@
logManager.log(waitLog);
for (PrimaryIndexOperationTracker primaryOpTracker : dsr.getOpTrackers()) {
// flush each partition one by one
- if (primaryOpTracker.getNumActiveOperations() > 0) {
- throw new IllegalStateException(
- "flushDatasetOpenIndexes is called on a dataset with currently active operations");
+ int numActiveOperations = primaryOpTracker.getNumActiveOperations();
+ if (numActiveOperations > 0) {
+ throw new IllegalStateException("flushDatasetOpenIndexes is called on dataset " + dsInfo.getDatasetID()
+ + " with currently " + "active operations, count=" + numActiveOperations);
}
primaryOpTracker.setFlushOnExit(true);
primaryOpTracker.flushIfNeeded();
diff --git a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/PrimaryIndexOperationTracker.java b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/PrimaryIndexOperationTracker.java
index ec05702..fb001a0 100644
--- a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/PrimaryIndexOperationTracker.java
+++ b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/PrimaryIndexOperationTracker.java
@@ -145,8 +145,8 @@
}
}
if (primaryLsmIndex == null) {
- throw new IllegalStateException(
- "Primary index not found in dataset " + dsInfo.getDatasetID() + " and partition " + partition);
+ throw new IllegalStateException("Primary index not found in dataset " + dsInfo.getDatasetID()
+ + " and partition " + partition + " open indexes " + indexes);
}
for (ILSMIndex lsmIndex : indexes) {
ILSMOperationTracker opTracker = lsmIndex.getOperationTracker();
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12923
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I57c05bc089527e745cf1851be0814242ba89ac55
Gerrit-Change-Number: 12923
Gerrit-PatchSet: 1
Gerrit-Owner: Ali Alsuliman <al...@gmail.com>
Gerrit-MessageType: newchange
Change in asterixdb[master]: [NO ISSUE][STO] Improve logging
Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
From Ali Alsuliman <al...@gmail.com>:
Ali Alsuliman has submitted this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12923 )
Change subject: [NO ISSUE][STO] Improve logging
......................................................................
[NO ISSUE][STO] Improve logging
- user model changes: no
- storage format changes: no
- interface changes: no
Change-Id: I57c05bc089527e745cf1851be0814242ba89ac55
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12923
Reviewed-by: Murtadha Hubail <mh...@apache.org>
Tested-by: Jenkins <je...@fulliautomatix.ics.uci.edu>
Integration-Tests: Jenkins <je...@fulliautomatix.ics.uci.edu>
---
M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/PrimaryIndexOperationTracker.java
2 files changed, 14 insertions(+), 7 deletions(-)
Approvals:
Murtadha Hubail: Looks good to me, approved
Jenkins: Verified; Verified
Objections:
Anon. E. Moose #1000171: Violations found
diff --git a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
index 2fe100e..bd24696 100644
--- a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
+++ b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
@@ -170,9 +170,15 @@
closeIndex(iInfo);
dsInfo.removeIndex(resourceID);
synchronized (dsInfo) {
- if (dsInfo.getReferenceCount() == 0 && dsInfo.isOpen() && dsInfo.getIndexes().isEmpty()
- && !dsInfo.isExternal()) {
+ int referenceCount = dsInfo.getReferenceCount();
+ boolean open = dsInfo.isOpen();
+ boolean empty = dsInfo.getIndexes().isEmpty();
+ if (referenceCount == 0 && open && empty && !dsInfo.isExternal()) {
+ LOGGER.debug("removing dataset {} from cache", dsInfo.getDatasetID());
removeDatasetFromCache(dsInfo.getDatasetID());
+ } else {
+ LOGGER.debug("keeping dataset {} in cache, ref count {}, open {}, indexes count: {}",
+ dsInfo.getDatasetID(), referenceCount, open, dsInfo.getIndexes().size());
}
}
}
@@ -414,9 +420,10 @@
logManager.log(waitLog);
for (PrimaryIndexOperationTracker primaryOpTracker : dsr.getOpTrackers()) {
// flush each partition one by one
- if (primaryOpTracker.getNumActiveOperations() > 0) {
- throw new IllegalStateException(
- "flushDatasetOpenIndexes is called on a dataset with currently active operations");
+ int numActiveOperations = primaryOpTracker.getNumActiveOperations();
+ if (numActiveOperations > 0) {
+ throw new IllegalStateException("flushDatasetOpenIndexes is called on dataset " + dsInfo.getDatasetID()
+ + " with currently " + "active operations, count=" + numActiveOperations);
}
primaryOpTracker.setFlushOnExit(true);
primaryOpTracker.flushIfNeeded();
diff --git a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/PrimaryIndexOperationTracker.java b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/PrimaryIndexOperationTracker.java
index ec05702..fb001a0 100644
--- a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/PrimaryIndexOperationTracker.java
+++ b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/PrimaryIndexOperationTracker.java
@@ -145,8 +145,8 @@
}
}
if (primaryLsmIndex == null) {
- throw new IllegalStateException(
- "Primary index not found in dataset " + dsInfo.getDatasetID() + " and partition " + partition);
+ throw new IllegalStateException("Primary index not found in dataset " + dsInfo.getDatasetID()
+ + " and partition " + partition + " open indexes " + indexes);
}
for (ILSMIndex lsmIndex : indexes) {
ILSMOperationTracker opTracker = lsmIndex.getOperationTracker();
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12923
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I57c05bc089527e745cf1851be0814242ba89ac55
Gerrit-Change-Number: 12923
Gerrit-PatchSet: 2
Gerrit-Owner: Ali Alsuliman <al...@gmail.com>
Gerrit-Reviewer: Ali Alsuliman <al...@gmail.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Murtadha Hubail <mh...@apache.org>
Gerrit-MessageType: merged
Change in asterixdb[master]: [NO ISSUE][STO] Improve logging
Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
From Ali Alsuliman <al...@gmail.com>:
Ali Alsuliman has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12923 )
Change subject: [NO ISSUE][STO] Improve logging
......................................................................
[NO ISSUE][STO] Improve logging
- user model changes: no
- storage format changes: no
- interface changes: no
Change-Id: I57c05bc089527e745cf1851be0814242ba89ac55
---
M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/PrimaryIndexOperationTracker.java
2 files changed, 14 insertions(+), 7 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/23/12923/1
diff --git a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
index 2fe100e..bd24696 100644
--- a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
+++ b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
@@ -170,9 +170,15 @@
closeIndex(iInfo);
dsInfo.removeIndex(resourceID);
synchronized (dsInfo) {
- if (dsInfo.getReferenceCount() == 0 && dsInfo.isOpen() && dsInfo.getIndexes().isEmpty()
- && !dsInfo.isExternal()) {
+ int referenceCount = dsInfo.getReferenceCount();
+ boolean open = dsInfo.isOpen();
+ boolean empty = dsInfo.getIndexes().isEmpty();
+ if (referenceCount == 0 && open && empty && !dsInfo.isExternal()) {
+ LOGGER.debug("removing dataset {} from cache", dsInfo.getDatasetID());
removeDatasetFromCache(dsInfo.getDatasetID());
+ } else {
+ LOGGER.debug("keeping dataset {} in cache, ref count {}, open {}, indexes count: {}",
+ dsInfo.getDatasetID(), referenceCount, open, dsInfo.getIndexes().size());
}
}
}
@@ -414,9 +420,10 @@
logManager.log(waitLog);
for (PrimaryIndexOperationTracker primaryOpTracker : dsr.getOpTrackers()) {
// flush each partition one by one
- if (primaryOpTracker.getNumActiveOperations() > 0) {
- throw new IllegalStateException(
- "flushDatasetOpenIndexes is called on a dataset with currently active operations");
+ int numActiveOperations = primaryOpTracker.getNumActiveOperations();
+ if (numActiveOperations > 0) {
+ throw new IllegalStateException("flushDatasetOpenIndexes is called on dataset " + dsInfo.getDatasetID()
+ + " with currently " + "active operations, count=" + numActiveOperations);
}
primaryOpTracker.setFlushOnExit(true);
primaryOpTracker.flushIfNeeded();
diff --git a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/PrimaryIndexOperationTracker.java b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/PrimaryIndexOperationTracker.java
index ec05702..fb001a0 100644
--- a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/PrimaryIndexOperationTracker.java
+++ b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/PrimaryIndexOperationTracker.java
@@ -145,8 +145,8 @@
}
}
if (primaryLsmIndex == null) {
- throw new IllegalStateException(
- "Primary index not found in dataset " + dsInfo.getDatasetID() + " and partition " + partition);
+ throw new IllegalStateException("Primary index not found in dataset " + dsInfo.getDatasetID()
+ + " and partition " + partition + " open indexes " + indexes);
}
for (ILSMIndex lsmIndex : indexes) {
ILSMOperationTracker opTracker = lsmIndex.getOperationTracker();
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12923
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I57c05bc089527e745cf1851be0814242ba89ac55
Gerrit-Change-Number: 12923
Gerrit-PatchSet: 1
Gerrit-Owner: Ali Alsuliman <al...@gmail.com>
Gerrit-MessageType: newchange
Change in asterixdb[master]: [NO ISSUE][STO] Improve logging
Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
From Jenkins <je...@fulliautomatix.ics.uci.edu>:
Jenkins has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12923 )
Change subject: [NO ISSUE][STO] Improve logging
......................................................................
Patch Set 1: Integration-Tests+1
Integration Tests Successful
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/12410/ : SUCCESS
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12923
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I57c05bc089527e745cf1851be0814242ba89ac55
Gerrit-Change-Number: 12923
Gerrit-PatchSet: 1
Gerrit-Owner: Ali Alsuliman <al...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Murtadha Hubail <mh...@apache.org>
Gerrit-CC: Anon. E. Moose #1000171
Gerrit-Comment-Date: Sun, 22 Aug 2021 02:27:32 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: Yes
Gerrit-MessageType: comment
Change in asterixdb[master]: [NO ISSUE][STO] Improve logging
Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12923 )
Change subject: [NO ISSUE][STO] Improve logging
......................................................................
Patch Set 1:
Analytics Compatibility Compilation Successful
https://cbjenkins.page.link/ndkaivN6NNu9WakP9 : SUCCESS
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12923
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I57c05bc089527e745cf1851be0814242ba89ac55
Gerrit-Change-Number: 12923
Gerrit-PatchSet: 1
Gerrit-Owner: Ali Alsuliman <al...@gmail.com>
Gerrit-CC: Anon. E. Moose #1000171
Gerrit-CC: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Comment-Date: Sun, 22 Aug 2021 00:45:09 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: No
Gerrit-MessageType: comment