You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by cs...@apache.org on 2019/10/15 11:01:08 UTC
[impala] branch master updated (f65c2a7 -> f41a188)
This is an automated email from the ASF dual-hosted git repository.
csringhofer pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git.
from f65c2a7 IMPALA-8498: Write column index for floating types when NaN is not present
new b0b8f20 IMPALA-8950: Set overwrite to False by default in fs_client.copy
new f41a188 IMPALA-9008: Serialize Maven invocations to deflake query-event-hook-api
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
query-event-hook-api/CMakeLists.txt | 4 +++-
tests/util/hdfs_util.py | 2 +-
2 files changed, 4 insertions(+), 2 deletions(-)
[impala] 02/02: IMPALA-9008: Serialize Maven invocations to deflake
query-event-hook-api
Posted by cs...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
csringhofer pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit f41a1887fd85f526d32ef917c3ba94c6902e7e98
Author: Csaba Ringhofer <cs...@cloudera.com>
AuthorDate: Mon Oct 14 17:22:38 2019 +0200
IMPALA-9008: Serialize Maven invocations to deflake query-event-hook-api
The previous commit for this Jira serialized the compilation of
shaded-deps and ext-data-source to fix flakiness. It looks like
this led to a new similar issue related to the parallel compilation
of ext-data-source and query-event-hook-api.
Similarly to the shaded-deps issue, I try to fix this by adding a
fictional dependency (query-event-hook-api -> yarn-extras) to serialize
the build.
Full Maven dependency graph:
fe -> query-event-hook-api -> yarn-extras -> ext-data-source ->
shaded-deps-> impala-parent
The ordering of query-event-hook-apit, yarn-extras, ext-data-source,
and shaded-deps are arbitrary.
Change-Id: Ice0dfda5ee0ef60c9c208f05b0952555770fc090
Reviewed-on: http://gerrit.cloudera.org:8080/14423
Reviewed-by: Joe McDonnell <jo...@cloudera.com>
Reviewed-by: Csaba Ringhofer <cs...@cloudera.com>
Tested-by: Csaba Ringhofer <cs...@cloudera.com>
---
query-event-hook-api/CMakeLists.txt | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/query-event-hook-api/CMakeLists.txt b/query-event-hook-api/CMakeLists.txt
index 700e614..7352ed2 100644
--- a/query-event-hook-api/CMakeLists.txt
+++ b/query-event-hook-api/CMakeLists.txt
@@ -15,6 +15,8 @@
# specific language governing permissions and limitations
# under the License.
-add_custom_target(query-event-hook-api ALL DEPENDS gen-deps impala-parent
+# The dependency on yarn-extras is only added to avoid parallel downloads
+# of dependencies. For more details see IMPALA-7051, which was a similar issue.
+add_custom_target(query-event-hook-api ALL DEPENDS gen-deps impala-parent yarn-extras
COMMAND $ENV{IMPALA_HOME}/bin/mvn-quiet.sh -B install -DskipTests
)
[impala] 01/02: IMPALA-8950: Set overwrite to False by default in
fs_client.copy
Posted by cs...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
csringhofer pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit b0b8f20267f96c902eee5bab85475ee41645bbe3
Author: Sahil Takiar <ta...@gmail.com>
AuthorDate: Thu Oct 10 09:13:21 2019 -0700
IMPALA-8950: Set overwrite to False by default in fs_client.copy
test_udfs.py and test_load.py are consistently failing on S3 with
the following error:
E TypeError: copy() takes exactly 4 arguments (3 given)
The issue is that a new parameter 'overwrite' was added to the method
'HadoopFsCommandLineClient.copy', but with no default value. This patch
fixes the issue by adding 'overwrite=False'.
Testing:
* Ran core tests on S3
Change-Id: I1935e0c92f7402e727ee16596485147cf1a5a516
Reviewed-on: http://gerrit.cloudera.org:8080/14432
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
tests/util/hdfs_util.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tests/util/hdfs_util.py b/tests/util/hdfs_util.py
index b22dec4..81d07e6 100644
--- a/tests/util/hdfs_util.py
+++ b/tests/util/hdfs_util.py
@@ -226,7 +226,7 @@ class HadoopFsCommandLineClient(BaseFilesystem):
(status, stdout, stderr) = self._hadoop_fs_shell(['-mkdir', '-p', fixed_path])
return status == 0
- def copy(self, src, dst, overwrite):
+ def copy(self, src, dst, overwrite=False):
"""Copy the source file to the destination. Specifes the '-d' option by default, which
'Skip[s] creation of temporary file with the suffix ._COPYING_.' to avoid extraneous
copies on S3. If overwrite is true, the destination file is overwritten, set to false