You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@asterixdb.apache.org by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu> on 2021/08/22 00:38:24 UTC

Change in asterixdb[master]: [NO ISSUE][STO] Improve logging

From Ali Alsuliman <al...@gmail.com>:

Ali Alsuliman has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12923 )


Change subject: [NO ISSUE][STO] Improve logging
......................................................................

[NO ISSUE][STO] Improve logging

- user model changes: no
- storage format changes: no
- interface changes: no

Change-Id: I57c05bc089527e745cf1851be0814242ba89ac55
---
M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/PrimaryIndexOperationTracker.java
2 files changed, 14 insertions(+), 7 deletions(-)



  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/23/12923/1

diff --git a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
index 2fe100e..bd24696 100644
--- a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
+++ b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
@@ -170,9 +170,15 @@
         closeIndex(iInfo);
         dsInfo.removeIndex(resourceID);
         synchronized (dsInfo) {
-            if (dsInfo.getReferenceCount() == 0 && dsInfo.isOpen() && dsInfo.getIndexes().isEmpty()
-                    && !dsInfo.isExternal()) {
+            int referenceCount = dsInfo.getReferenceCount();
+            boolean open = dsInfo.isOpen();
+            boolean empty = dsInfo.getIndexes().isEmpty();
+            if (referenceCount == 0 && open && empty && !dsInfo.isExternal()) {
+                LOGGER.debug("removing dataset {} from cache", dsInfo.getDatasetID());
                 removeDatasetFromCache(dsInfo.getDatasetID());
+            } else {
+                LOGGER.debug("keeping dataset {} in cache, ref count {}, open {}, indexes count: {}",
+                        dsInfo.getDatasetID(), referenceCount, open, dsInfo.getIndexes().size());
             }
         }
     }
@@ -414,9 +420,10 @@
         logManager.log(waitLog);
         for (PrimaryIndexOperationTracker primaryOpTracker : dsr.getOpTrackers()) {
             // flush each partition one by one
-            if (primaryOpTracker.getNumActiveOperations() > 0) {
-                throw new IllegalStateException(
-                        "flushDatasetOpenIndexes is called on a dataset with currently active operations");
+            int numActiveOperations = primaryOpTracker.getNumActiveOperations();
+            if (numActiveOperations > 0) {
+                throw new IllegalStateException("flushDatasetOpenIndexes is called on dataset " + dsInfo.getDatasetID()
+                        + " with currently " + "active operations, count=" + numActiveOperations);
             }
             primaryOpTracker.setFlushOnExit(true);
             primaryOpTracker.flushIfNeeded();
diff --git a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/PrimaryIndexOperationTracker.java b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/PrimaryIndexOperationTracker.java
index ec05702..fb001a0 100644
--- a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/PrimaryIndexOperationTracker.java
+++ b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/PrimaryIndexOperationTracker.java
@@ -145,8 +145,8 @@
                 }
             }
             if (primaryLsmIndex == null) {
-                throw new IllegalStateException(
-                        "Primary index not found in dataset " + dsInfo.getDatasetID() + " and partition " + partition);
+                throw new IllegalStateException("Primary index not found in dataset " + dsInfo.getDatasetID()
+                        + " and partition " + partition + " open indexes " + indexes);
             }
             for (ILSMIndex lsmIndex : indexes) {
                 ILSMOperationTracker opTracker = lsmIndex.getOperationTracker();

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12923
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I57c05bc089527e745cf1851be0814242ba89ac55
Gerrit-Change-Number: 12923
Gerrit-PatchSet: 1
Gerrit-Owner: Ali Alsuliman <al...@gmail.com>
Gerrit-MessageType: newchange

Change in asterixdb[master]: [NO ISSUE][STO] Improve logging

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
From Ali Alsuliman <al...@gmail.com>:

Ali Alsuliman has submitted this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12923 )

Change subject: [NO ISSUE][STO] Improve logging
......................................................................

[NO ISSUE][STO] Improve logging

- user model changes: no
- storage format changes: no
- interface changes: no

Change-Id: I57c05bc089527e745cf1851be0814242ba89ac55
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12923
Reviewed-by: Murtadha Hubail <mh...@apache.org>
Tested-by: Jenkins <je...@fulliautomatix.ics.uci.edu>
Integration-Tests: Jenkins <je...@fulliautomatix.ics.uci.edu>
---
M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/PrimaryIndexOperationTracker.java
2 files changed, 14 insertions(+), 7 deletions(-)

Approvals:
  Murtadha Hubail: Looks good to me, approved
  Jenkins: Verified; Verified

Objections:
  Anon. E. Moose #1000171: Violations found



diff --git a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
index 2fe100e..bd24696 100644
--- a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
+++ b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
@@ -170,9 +170,15 @@
         closeIndex(iInfo);
         dsInfo.removeIndex(resourceID);
         synchronized (dsInfo) {
-            if (dsInfo.getReferenceCount() == 0 && dsInfo.isOpen() && dsInfo.getIndexes().isEmpty()
-                    && !dsInfo.isExternal()) {
+            int referenceCount = dsInfo.getReferenceCount();
+            boolean open = dsInfo.isOpen();
+            boolean empty = dsInfo.getIndexes().isEmpty();
+            if (referenceCount == 0 && open && empty && !dsInfo.isExternal()) {
+                LOGGER.debug("removing dataset {} from cache", dsInfo.getDatasetID());
                 removeDatasetFromCache(dsInfo.getDatasetID());
+            } else {
+                LOGGER.debug("keeping dataset {} in cache, ref count {}, open {}, indexes count: {}",
+                        dsInfo.getDatasetID(), referenceCount, open, dsInfo.getIndexes().size());
             }
         }
     }
@@ -414,9 +420,10 @@
         logManager.log(waitLog);
         for (PrimaryIndexOperationTracker primaryOpTracker : dsr.getOpTrackers()) {
             // flush each partition one by one
-            if (primaryOpTracker.getNumActiveOperations() > 0) {
-                throw new IllegalStateException(
-                        "flushDatasetOpenIndexes is called on a dataset with currently active operations");
+            int numActiveOperations = primaryOpTracker.getNumActiveOperations();
+            if (numActiveOperations > 0) {
+                throw new IllegalStateException("flushDatasetOpenIndexes is called on dataset " + dsInfo.getDatasetID()
+                        + " with currently " + "active operations, count=" + numActiveOperations);
             }
             primaryOpTracker.setFlushOnExit(true);
             primaryOpTracker.flushIfNeeded();
diff --git a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/PrimaryIndexOperationTracker.java b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/PrimaryIndexOperationTracker.java
index ec05702..fb001a0 100644
--- a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/PrimaryIndexOperationTracker.java
+++ b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/PrimaryIndexOperationTracker.java
@@ -145,8 +145,8 @@
                 }
             }
             if (primaryLsmIndex == null) {
-                throw new IllegalStateException(
-                        "Primary index not found in dataset " + dsInfo.getDatasetID() + " and partition " + partition);
+                throw new IllegalStateException("Primary index not found in dataset " + dsInfo.getDatasetID()
+                        + " and partition " + partition + " open indexes " + indexes);
             }
             for (ILSMIndex lsmIndex : indexes) {
                 ILSMOperationTracker opTracker = lsmIndex.getOperationTracker();

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12923
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I57c05bc089527e745cf1851be0814242ba89ac55
Gerrit-Change-Number: 12923
Gerrit-PatchSet: 2
Gerrit-Owner: Ali Alsuliman <al...@gmail.com>
Gerrit-Reviewer: Ali Alsuliman <al...@gmail.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Murtadha Hubail <mh...@apache.org>
Gerrit-MessageType: merged

Change in asterixdb[master]: [NO ISSUE][STO] Improve logging

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
From Ali Alsuliman <al...@gmail.com>:

Ali Alsuliman has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12923 )


Change subject: [NO ISSUE][STO] Improve logging
......................................................................

[NO ISSUE][STO] Improve logging

- user model changes: no
- storage format changes: no
- interface changes: no

Change-Id: I57c05bc089527e745cf1851be0814242ba89ac55
---
M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/PrimaryIndexOperationTracker.java
2 files changed, 14 insertions(+), 7 deletions(-)



  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/23/12923/1

diff --git a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
index 2fe100e..bd24696 100644
--- a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
+++ b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
@@ -170,9 +170,15 @@
         closeIndex(iInfo);
         dsInfo.removeIndex(resourceID);
         synchronized (dsInfo) {
-            if (dsInfo.getReferenceCount() == 0 && dsInfo.isOpen() && dsInfo.getIndexes().isEmpty()
-                    && !dsInfo.isExternal()) {
+            int referenceCount = dsInfo.getReferenceCount();
+            boolean open = dsInfo.isOpen();
+            boolean empty = dsInfo.getIndexes().isEmpty();
+            if (referenceCount == 0 && open && empty && !dsInfo.isExternal()) {
+                LOGGER.debug("removing dataset {} from cache", dsInfo.getDatasetID());
                 removeDatasetFromCache(dsInfo.getDatasetID());
+            } else {
+                LOGGER.debug("keeping dataset {} in cache, ref count {}, open {}, indexes count: {}",
+                        dsInfo.getDatasetID(), referenceCount, open, dsInfo.getIndexes().size());
             }
         }
     }
@@ -414,9 +420,10 @@
         logManager.log(waitLog);
         for (PrimaryIndexOperationTracker primaryOpTracker : dsr.getOpTrackers()) {
             // flush each partition one by one
-            if (primaryOpTracker.getNumActiveOperations() > 0) {
-                throw new IllegalStateException(
-                        "flushDatasetOpenIndexes is called on a dataset with currently active operations");
+            int numActiveOperations = primaryOpTracker.getNumActiveOperations();
+            if (numActiveOperations > 0) {
+                throw new IllegalStateException("flushDatasetOpenIndexes is called on dataset " + dsInfo.getDatasetID()
+                        + " with currently " + "active operations, count=" + numActiveOperations);
             }
             primaryOpTracker.setFlushOnExit(true);
             primaryOpTracker.flushIfNeeded();
diff --git a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/PrimaryIndexOperationTracker.java b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/PrimaryIndexOperationTracker.java
index ec05702..fb001a0 100644
--- a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/PrimaryIndexOperationTracker.java
+++ b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/PrimaryIndexOperationTracker.java
@@ -145,8 +145,8 @@
                 }
             }
             if (primaryLsmIndex == null) {
-                throw new IllegalStateException(
-                        "Primary index not found in dataset " + dsInfo.getDatasetID() + " and partition " + partition);
+                throw new IllegalStateException("Primary index not found in dataset " + dsInfo.getDatasetID()
+                        + " and partition " + partition + " open indexes " + indexes);
             }
             for (ILSMIndex lsmIndex : indexes) {
                 ILSMOperationTracker opTracker = lsmIndex.getOperationTracker();

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12923
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I57c05bc089527e745cf1851be0814242ba89ac55
Gerrit-Change-Number: 12923
Gerrit-PatchSet: 1
Gerrit-Owner: Ali Alsuliman <al...@gmail.com>
Gerrit-MessageType: newchange

Change in asterixdb[master]: [NO ISSUE][STO] Improve logging

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
From Jenkins <je...@fulliautomatix.ics.uci.edu>:

Jenkins has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12923 )

Change subject: [NO ISSUE][STO] Improve logging
......................................................................


Patch Set 1: Integration-Tests+1

Integration Tests Successful

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/12410/ : SUCCESS


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12923
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I57c05bc089527e745cf1851be0814242ba89ac55
Gerrit-Change-Number: 12923
Gerrit-PatchSet: 1
Gerrit-Owner: Ali Alsuliman <al...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Murtadha Hubail <mh...@apache.org>
Gerrit-CC: Anon. E. Moose #1000171
Gerrit-Comment-Date: Sun, 22 Aug 2021 02:27:32 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: Yes
Gerrit-MessageType: comment

Change in asterixdb[master]: [NO ISSUE][STO] Improve logging

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12923 )

Change subject: [NO ISSUE][STO] Improve logging
......................................................................


Patch Set 1:

Analytics Compatibility Compilation Successful
https://cbjenkins.page.link/ndkaivN6NNu9WakP9 : SUCCESS


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12923
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I57c05bc089527e745cf1851be0814242ba89ac55
Gerrit-Change-Number: 12923
Gerrit-PatchSet: 1
Gerrit-Owner: Ali Alsuliman <al...@gmail.com>
Gerrit-CC: Anon. E. Moose #1000171
Gerrit-CC: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Comment-Date: Sun, 22 Aug 2021 00:45:09 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: No
Gerrit-MessageType: comment