[44/50] [abbrv] incubator-impala git commit: IMPALA-3517: S3: Get rid of hdfsDelete() warning message

[ta...@apache.org]

IMPALA-3517: S3: Get rid of hdfsDelete() warning message

In the patch for IMPALA-3452, a bug was introduced which does an
unnecessary call to hdfsDelete() on the staging directory of a
partition(s), when the user selects the option of skipping the
staging step. Also, this delete will not happen when it should happen,
e.g on HDFS partitions. (Although it will be deleted by the
coordinator later).

This patch fixes this bug by making sure that we delete the staging
directory when it is necessary and skip the call to delete when we
do not want it.

Change-Id: I0a81ba0abfc24ee56689211579f46ac353e98adb
Reviewed-on: http://gerrit.cloudera.org:8080/3019
Reviewed-by: Sailesh Mukil <sa...@cloudera.com>
Tested-by: Internal Jenkins


Project: http://git-wip-us.apache.org/repos/asf/incubator-impala/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-impala/commit/da49a37e
Tree: http://git-wip-us.apache.org/repos/asf/incubator-impala/tree/da49a37e
Diff: http://git-wip-us.apache.org/repos/asf/incubator-impala/diff/da49a37e

Branch: refs/heads/master
Commit: da49a37edc136bd97a7255a57325ede8d37e6b5e
Parents: 5c56ec0
Author: Sailesh Mukil <sa...@cloudera.com>
Authored: Tue May 10 15:27:23 2016 -0700
Committer: Tim Armstrong <ta...@cloudera.com>
Committed: Thu May 12 14:18:03 2016 -0700

----------------------------------------------------------------------
 be/src/exec/hdfs-table-sink.cc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/da49a37e/be/src/exec/hdfs-table-sink.cc
----------------------------------------------------------------------
diff --git a/be/src/exec/hdfs-table-sink.cc b/be/src/exec/hdfs-table-sink.cc
index d3a2ade..0a935eb 100644
--- a/be/src/exec/hdfs-table-sink.cc
+++ b/be/src/exec/hdfs-table-sink.cc
@@ -506,7 +506,7 @@ inline Status HdfsTableSink::GetOutputPartition(RuntimeState* state,
     state->per_partition_status()->insert(
         make_pair(partition->partition_name, partition_status));
 
-    if (!no_more_rows && ShouldSkipStaging(state, partition)) {
+    if (!no_more_rows && !ShouldSkipStaging(state, partition)) {
       // Indicate that temporary directory is to be deleted after execution
       (*state->hdfs_files_to_move())[partition->tmp_hdfs_dir_name] = "";
     }