You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@asterixdb.apache.org by "Murtadha Hubail (Code Review)" <do...@asterixdb.incubator.apache.org> on 2015/09/26 11:53:32 UTC
Change in asterixdb[master]: ASTERIXDB-1114: Fix missing DatasetInfo in IndexOperationTra...
Murtadha Hubail has uploaded a new change for review.
https://asterix-gerrit.ics.uci.edu/411
Change subject: ASTERIXDB-1114: Fix missing DatasetInfo in IndexOperationTracker
......................................................................
ASTERIXDB-1114: Fix missing DatasetInfo in IndexOperationTracker
Change-Id: I28ae228e4322c07128a40432378ecaace913ad80
---
M asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
1 file changed, 18 insertions(+), 16 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/11/411/1
diff --git a/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java b/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
index 27fb2c6..9df4970 100644
--- a/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
+++ b/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
@@ -94,16 +94,15 @@
int did = getDIDfromRID(resourceID);
DatasetInfo dsInfo = datasetInfos.get(did);
if (dsInfo == null) {
- dsInfo = new DatasetInfo(did, !index.hasMemoryComponents());
- PrimaryIndexOperationTracker opTracker = (PrimaryIndexOperationTracker) datasetOpTrackers
- .get(dsInfo.datasetID);
- if (opTracker != null) {
- opTracker.setDatasetInfo(dsInfo);
- }
- } else if (dsInfo.indexes.containsKey(resourceID)) {
+ dsInfo = getDatasetInfo(did);
+ }
+ if (!dsInfo.isRegistered) {
+ dsInfo.isExternal = !index.hasMemoryComponents();
+ dsInfo.isRegistered = true;
+ }
+ if (dsInfo.indexes.containsKey(resourceID)) {
throw new HyracksDataException("Index with resource ID " + resourceID + " already exists.");
}
- datasetInfos.put(did, dsInfo);
dsInfo.indexes.put(resourceID, new IndexInfo((ILSMIndex) index));
}
@@ -131,7 +130,6 @@
}
// TODO: use fine-grained counters, one for each index instead of a single counter per dataset.
-
// First wait for any ongoing IO operations
synchronized (dsInfo) {
while (dsInfo.numActiveIOOps > 0) {
@@ -172,7 +170,7 @@
public synchronized void open(long resourceID) throws HyracksDataException {
int did = getDIDfromRID(resourceID);
DatasetInfo dsInfo = datasetInfos.get(did);
- if (dsInfo == null) {
+ if (dsInfo == null || !dsInfo.isRegistered) {
throw new HyracksDataException("Failed to open index with resource ID " + resourceID
+ " since it does not exist.");
}
@@ -251,9 +249,13 @@
}
public DatasetInfo getDatasetInfo(int datasetID) {
-
synchronized (datasetInfos) {
- return datasetInfos.get(datasetID);
+ DatasetInfo dsInfo = datasetInfos.get(datasetID);
+ if (dsInfo == null) {
+ dsInfo = new DatasetInfo(datasetID);
+ datasetInfos.put(datasetID, dsInfo);
+ }
+ return dsInfo;
}
}
@@ -346,13 +348,14 @@
private final int datasetID;
private long lastAccess;
private int numActiveIOOps;
- private final boolean isExternal;
+ private boolean isExternal;
+ private boolean isRegistered;
- public DatasetInfo(int datasetID, boolean isExternal) {
+ public DatasetInfo(int datasetID) {
this.indexes = new HashMap<Long, IndexInfo>();
this.lastAccess = -1;
this.datasetID = datasetID;
- this.isExternal = isExternal;
+ this.isRegistered = false;
}
@Override
@@ -514,7 +517,6 @@
// TODO: This is not efficient since we flush the indexes sequentially.
// Think of a way to allow submitting the flush requests concurrently. We don't do them concurrently because this
// may lead to a deadlock scenario between the DatasetLifeCycleManager and the PrimaryIndexOperationTracker.
-
flushAndWaitForIO(dsInfo, iInfo);
}
}
--
To view, visit https://asterix-gerrit.ics.uci.edu/411
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I28ae228e4322c07128a40432378ecaace913ad80
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail <hu...@gmail.com>
Change in asterixdb[master]: ASTERIXDB-1114: Fix missing DatasetInfo in IndexOperationTra...
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: ASTERIXDB-1114: Fix missing DatasetInfo in IndexOperationTracker
......................................................................
Patch Set 1: Verified+1
Build Successful
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/303/ : SUCCESS
--
To view, visit https://asterix-gerrit.ics.uci.edu/411
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I28ae228e4322c07128a40432378ecaace913ad80
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail <hu...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: ASTERIXDB-1114: Fix missing DatasetInfo in IndexOperationTra...
Posted by "Murtadha Hubail (Code Review)" <do...@asterixdb.incubator.apache.org>.
Murtadha Hubail has submitted this change and it was merged.
Change subject: ASTERIXDB-1114: Fix missing DatasetInfo in IndexOperationTracker
......................................................................
ASTERIXDB-1114: Fix missing DatasetInfo in IndexOperationTracker
Change-Id: I28ae228e4322c07128a40432378ecaace913ad80
Reviewed-on: https://asterix-gerrit.ics.uci.edu/411
Tested-by: Jenkins <je...@fulliautomatix.ics.uci.edu>
Reviewed-by: Young-Seok Kim <ki...@gmail.com>
---
M asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
1 file changed, 18 insertions(+), 16 deletions(-)
Approvals:
Young-Seok Kim: Looks good to me, approved
Jenkins: Verified
diff --git a/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java b/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
index 27fb2c6..9df4970 100644
--- a/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
+++ b/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
@@ -94,16 +94,15 @@
int did = getDIDfromRID(resourceID);
DatasetInfo dsInfo = datasetInfos.get(did);
if (dsInfo == null) {
- dsInfo = new DatasetInfo(did, !index.hasMemoryComponents());
- PrimaryIndexOperationTracker opTracker = (PrimaryIndexOperationTracker) datasetOpTrackers
- .get(dsInfo.datasetID);
- if (opTracker != null) {
- opTracker.setDatasetInfo(dsInfo);
- }
- } else if (dsInfo.indexes.containsKey(resourceID)) {
+ dsInfo = getDatasetInfo(did);
+ }
+ if (!dsInfo.isRegistered) {
+ dsInfo.isExternal = !index.hasMemoryComponents();
+ dsInfo.isRegistered = true;
+ }
+ if (dsInfo.indexes.containsKey(resourceID)) {
throw new HyracksDataException("Index with resource ID " + resourceID + " already exists.");
}
- datasetInfos.put(did, dsInfo);
dsInfo.indexes.put(resourceID, new IndexInfo((ILSMIndex) index));
}
@@ -131,7 +130,6 @@
}
// TODO: use fine-grained counters, one for each index instead of a single counter per dataset.
-
// First wait for any ongoing IO operations
synchronized (dsInfo) {
while (dsInfo.numActiveIOOps > 0) {
@@ -172,7 +170,7 @@
public synchronized void open(long resourceID) throws HyracksDataException {
int did = getDIDfromRID(resourceID);
DatasetInfo dsInfo = datasetInfos.get(did);
- if (dsInfo == null) {
+ if (dsInfo == null || !dsInfo.isRegistered) {
throw new HyracksDataException("Failed to open index with resource ID " + resourceID
+ " since it does not exist.");
}
@@ -251,9 +249,13 @@
}
public DatasetInfo getDatasetInfo(int datasetID) {
-
synchronized (datasetInfos) {
- return datasetInfos.get(datasetID);
+ DatasetInfo dsInfo = datasetInfos.get(datasetID);
+ if (dsInfo == null) {
+ dsInfo = new DatasetInfo(datasetID);
+ datasetInfos.put(datasetID, dsInfo);
+ }
+ return dsInfo;
}
}
@@ -346,13 +348,14 @@
private final int datasetID;
private long lastAccess;
private int numActiveIOOps;
- private final boolean isExternal;
+ private boolean isExternal;
+ private boolean isRegistered;
- public DatasetInfo(int datasetID, boolean isExternal) {
+ public DatasetInfo(int datasetID) {
this.indexes = new HashMap<Long, IndexInfo>();
this.lastAccess = -1;
this.datasetID = datasetID;
- this.isExternal = isExternal;
+ this.isRegistered = false;
}
@Override
@@ -514,7 +517,6 @@
// TODO: This is not efficient since we flush the indexes sequentially.
// Think of a way to allow submitting the flush requests concurrently. We don't do them concurrently because this
// may lead to a deadlock scenario between the DatasetLifeCycleManager and the PrimaryIndexOperationTracker.
-
flushAndWaitForIO(dsInfo, iInfo);
}
}
--
To view, visit https://asterix-gerrit.ics.uci.edu/411
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I28ae228e4322c07128a40432378ecaace913ad80
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail <hu...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Murtadha Hubail <hu...@gmail.com>
Gerrit-Reviewer: Young-Seok Kim <ki...@gmail.com>
Change in asterixdb[master]: ASTERIXDB-1114: Fix missing DatasetInfo in IndexOperationTra...
Posted by "Young-Seok Kim (Code Review)" <do...@asterixdb.incubator.apache.org>.
Young-Seok Kim has posted comments on this change.
Change subject: ASTERIXDB-1114: Fix missing DatasetInfo in IndexOperationTracker
......................................................................
Patch Set 1: Code-Review+2
--
To view, visit https://asterix-gerrit.ics.uci.edu/411
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I28ae228e4322c07128a40432378ecaace913ad80
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail <hu...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Young-Seok Kim <ki...@gmail.com>
Gerrit-HasComments: No
Change in asterixdb[master]: ASTERIXDB-1114: Fix missing DatasetInfo in IndexOperationTra...
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: ASTERIXDB-1114: Fix missing DatasetInfo in IndexOperationTracker
......................................................................
Patch Set 1:
Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/303/
--
To view, visit https://asterix-gerrit.ics.uci.edu/411
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I28ae228e4322c07128a40432378ecaace913ad80
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail <hu...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No