You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@asterixdb.apache.org by AsterixDB Code Review <do...@vitalstatistix.ics.uci.edu> on 2019/12/07 22:56:30 UTC
Change in asterixdb[stabilization-f69489]: [NO ISSUE][RT] Delete run files eagerly
From Ali Alsuliman <al...@gmail.com>:
Ali Alsuliman has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4403 )
Change subject: [NO ISSUE][RT] Delete run files eagerly
......................................................................
[NO ISSUE][RT] Delete run files eagerly
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
Delete run files eagerly.
Change-Id: Idc780b44b0554bf36e2378700aae1b38b8e57353
---
M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/group/external/ExternalGroupWriteOperatorNodePushable.java
M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java
M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractExternalSortRunMerger.java
3 files changed, 20 insertions(+), 3 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/03/4403/1
diff --git a/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/group/external/ExternalGroupWriteOperatorNodePushable.java b/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/group/external/ExternalGroupWriteOperatorNodePushable.java
index 999dbac..e8a1b76 100644
--- a/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/group/external/ExternalGroupWriteOperatorNodePushable.java
+++ b/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/group/external/ExternalGroupWriteOperatorNodePushable.java
@@ -171,8 +171,7 @@
@Override
public RunFileWriter getRunFileWriter() throws HyracksDataException {
- FileReference newRun = ctx.getJobletContext()
- .createManagedWorkspaceFile(ExternalGroupOperatorDescriptor.class.getSimpleName());
+ FileReference newRun = ctx.createManagedWorkspaceFile(ExternalGroupOperatorDescriptor.class.getSimpleName());
return new RunFileWriter(newRun, ctx.getIoManager());
}
}
diff --git a/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java b/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java
index e7984d6..6ba03c5 100644
--- a/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java
+++ b/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java
@@ -442,6 +442,12 @@
if (isLeftOuter && pReader != null) {
appendNullToProbeTuples(pReader);
}
+ if (bReader != null) {
+ bReader.close();
+ }
+ if (pReader != null) {
+ pReader.close();
+ }
continue;
}
int bSize = state.hybridHJ.getBuildPartitionSizeInTup(pid);
@@ -637,6 +643,12 @@
// For the outer join, we don't reverse the role.
appendNullToProbeTuples(rprfw);
}
+ if (rbrfw != null) {
+ rbrfw.close();
+ }
+ if (rprfw != null) {
+ rprfw.close();
+ }
continue;
}
@@ -661,6 +673,12 @@
// For the outer join, we don't reverse the role.
appendNullToProbeTuples(rprfw);
}
+ if (rbrfw != null) {
+ rbrfw.close();
+ }
+ if (rprfw != null) {
+ rprfw.close();
+ }
continue;
}
diff --git a/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractExternalSortRunMerger.java b/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractExternalSortRunMerger.java
index 4a77b3c..f3e9320 100644
--- a/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractExternalSortRunMerger.java
+++ b/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractExternalSortRunMerger.java
@@ -142,7 +142,7 @@
} finally {
mergeResultWriter.close();
}
- reader = mergeFileWriter.createReader();
+ reader = mergeFileWriter.createDeleteOnCloseReader();
}
runs.add(reader);
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4403
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: stabilization-f69489
Gerrit-Change-Id: Idc780b44b0554bf36e2378700aae1b38b8e57353
Gerrit-Change-Number: 4403
Gerrit-PatchSet: 1
Gerrit-Owner: Ali Alsuliman <al...@gmail.com>
Gerrit-MessageType: newchange
Change in asterixdb[stabilization-f69489]: [NO ISSUE][RT] Delete run files eagerly
Posted by AsterixDB Code Review <do...@vitalstatistix.ics.uci.edu>.
Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4403 )
Change subject: [NO ISSUE][RT] Delete run files eagerly
......................................................................
Patch Set 2:
Analytics Compatibility Compilation Successful
https://cbjenkins.page.link/2CVa : SUCCESS
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4403
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: stabilization-f69489
Gerrit-Change-Id: Idc780b44b0554bf36e2378700aae1b38b8e57353
Gerrit-Change-Number: 4403
Gerrit-PatchSet: 2
Gerrit-Owner: Ali Alsuliman <al...@gmail.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Comment-Date: Mon, 09 Dec 2019 05:09:49 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: No
Gerrit-MessageType: comment
Change in asterixdb[stabilization-f69489]: [NO ISSUE][RT] Delete run files eagerly
Posted by AsterixDB Code Review <do...@vitalstatistix.ics.uci.edu>.
From Ali Alsuliman <al...@gmail.com>:
Ali Alsuliman has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4403 )
Change subject: [NO ISSUE][RT] Delete run files eagerly
......................................................................
[NO ISSUE][RT] Delete run files eagerly
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
Delete run files eagerly.
Change-Id: Idc780b44b0554bf36e2378700aae1b38b8e57353
---
M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/group/external/ExternalGroupWriteOperatorNodePushable.java
M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java
M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractExternalSortRunMerger.java
3 files changed, 20 insertions(+), 3 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/03/4403/1
diff --git a/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/group/external/ExternalGroupWriteOperatorNodePushable.java b/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/group/external/ExternalGroupWriteOperatorNodePushable.java
index 999dbac..e8a1b76 100644
--- a/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/group/external/ExternalGroupWriteOperatorNodePushable.java
+++ b/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/group/external/ExternalGroupWriteOperatorNodePushable.java
@@ -171,8 +171,7 @@
@Override
public RunFileWriter getRunFileWriter() throws HyracksDataException {
- FileReference newRun = ctx.getJobletContext()
- .createManagedWorkspaceFile(ExternalGroupOperatorDescriptor.class.getSimpleName());
+ FileReference newRun = ctx.createManagedWorkspaceFile(ExternalGroupOperatorDescriptor.class.getSimpleName());
return new RunFileWriter(newRun, ctx.getIoManager());
}
}
diff --git a/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java b/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java
index e7984d6..6ba03c5 100644
--- a/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java
+++ b/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java
@@ -442,6 +442,12 @@
if (isLeftOuter && pReader != null) {
appendNullToProbeTuples(pReader);
}
+ if (bReader != null) {
+ bReader.close();
+ }
+ if (pReader != null) {
+ pReader.close();
+ }
continue;
}
int bSize = state.hybridHJ.getBuildPartitionSizeInTup(pid);
@@ -637,6 +643,12 @@
// For the outer join, we don't reverse the role.
appendNullToProbeTuples(rprfw);
}
+ if (rbrfw != null) {
+ rbrfw.close();
+ }
+ if (rprfw != null) {
+ rprfw.close();
+ }
continue;
}
@@ -661,6 +673,12 @@
// For the outer join, we don't reverse the role.
appendNullToProbeTuples(rprfw);
}
+ if (rbrfw != null) {
+ rbrfw.close();
+ }
+ if (rprfw != null) {
+ rprfw.close();
+ }
continue;
}
diff --git a/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractExternalSortRunMerger.java b/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractExternalSortRunMerger.java
index 4a77b3c..f3e9320 100644
--- a/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractExternalSortRunMerger.java
+++ b/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractExternalSortRunMerger.java
@@ -142,7 +142,7 @@
} finally {
mergeResultWriter.close();
}
- reader = mergeFileWriter.createReader();
+ reader = mergeFileWriter.createDeleteOnCloseReader();
}
runs.add(reader);
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4403
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: stabilization-f69489
Gerrit-Change-Id: Idc780b44b0554bf36e2378700aae1b38b8e57353
Gerrit-Change-Number: 4403
Gerrit-PatchSet: 1
Gerrit-Owner: Ali Alsuliman <al...@gmail.com>
Gerrit-MessageType: newchange
Change in asterixdb[stabilization-f69489]: [ASTERIXDB-2686][RT] Delete run files eagerly
Posted by AsterixDB Code Review <do...@vitalstatistix.ics.uci.edu>.
From Ali Alsuliman <al...@gmail.com>:
Hello Anon. E. Moose #1000171, Till Westmann, Jenkins, Michael Blow, Murtadha Hubail, Dmitry Lychagin,
I'd like you to reexamine a change. Please visit
https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4403
to look at the new patch set (#3).
Change subject: [ASTERIXDB-2686][RT] Delete run files eagerly
......................................................................
[ASTERIXDB-2686][RT] Delete run files eagerly
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
Delete run files eagerly.
Change-Id: Idc780b44b0554bf36e2378700aae1b38b8e57353
---
M hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/io/IWorkspaceFileFactory.java
M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/group/external/ExternalGroupWriteOperatorNodePushable.java
M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java
M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractExternalSortRunMerger.java
4 files changed, 36 insertions(+), 3 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/03/4403/3
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4403
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: stabilization-f69489
Gerrit-Change-Id: Idc780b44b0554bf36e2378700aae1b38b8e57353
Gerrit-Change-Number: 4403
Gerrit-PatchSet: 3
Gerrit-Owner: Ali Alsuliman <al...@gmail.com>
Gerrit-Reviewer: Ali Alsuliman <al...@gmail.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Michael Blow <mb...@apache.org>
Gerrit-Reviewer: Murtadha Hubail <mh...@apache.org>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-MessageType: newpatchset
Change in asterixdb[stabilization-f69489]: [NO ISSUE][RT] Delete run files eagerly
Posted by AsterixDB Code Review <do...@vitalstatistix.ics.uci.edu>.
From Ali Alsuliman <al...@gmail.com>:
Hello Anon. E. Moose #1000171, Jenkins,
I'd like you to reexamine a change. Please visit
https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4403
to look at the new patch set (#2).
Change subject: [NO ISSUE][RT] Delete run files eagerly
......................................................................
[NO ISSUE][RT] Delete run files eagerly
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
Delete run files eagerly.
Change-Id: Idc780b44b0554bf36e2378700aae1b38b8e57353
---
M hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/io/IWorkspaceFileFactory.java
M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/group/external/ExternalGroupWriteOperatorNodePushable.java
M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java
M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractExternalSortRunMerger.java
4 files changed, 36 insertions(+), 3 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/03/4403/2
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4403
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: stabilization-f69489
Gerrit-Change-Id: Idc780b44b0554bf36e2378700aae1b38b8e57353
Gerrit-Change-Number: 4403
Gerrit-PatchSet: 2
Gerrit-Owner: Ali Alsuliman <al...@gmail.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-MessageType: newpatchset
Change in asterixdb[stabilization-f69489]: [ASTERIXDB-2686][RT] Delete run files eagerly
Posted by AsterixDB Code Review <do...@vitalstatistix.ics.uci.edu>.
Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4403 )
Change subject: [ASTERIXDB-2686][RT] Delete run files eagerly
......................................................................
Patch Set 3: Contrib+1
Analytics Compatibility Tests Successful
https://cbjenkins.page.link/gonx : SUCCESS
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4403
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: stabilization-f69489
Gerrit-Change-Id: Idc780b44b0554bf36e2378700aae1b38b8e57353
Gerrit-Change-Number: 4403
Gerrit-PatchSet: 3
Gerrit-Owner: Ali Alsuliman <al...@gmail.com>
Gerrit-Reviewer: Ali Alsuliman <al...@gmail.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Michael Blow <mb...@apache.org>
Gerrit-Reviewer: Murtadha Hubail <mh...@apache.org>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-Comment-Date: Mon, 09 Dec 2019 22:24:15 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: Yes
Gerrit-MessageType: comment
Change in asterixdb[stabilization-f69489]: [NO ISSUE][RT] Delete run files eagerly
Posted by AsterixDB Code Review <do...@vitalstatistix.ics.uci.edu>.
Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4403 )
Change subject: [NO ISSUE][RT] Delete run files eagerly
......................................................................
Patch Set 1: Contrib+1
Analytics Compatibility Tests Successful
https://cbjenkins.page.link/ymez : SUCCESS
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4403
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: stabilization-f69489
Gerrit-Change-Id: Idc780b44b0554bf36e2378700aae1b38b8e57353
Gerrit-Change-Number: 4403
Gerrit-PatchSet: 1
Gerrit-Owner: Ali Alsuliman <al...@gmail.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Comment-Date: Sun, 08 Dec 2019 00:27:40 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: Yes
Gerrit-MessageType: comment