You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Alex Behm (Code Review)" <ge...@cloudera.org> on 2018/04/24 16:39:03 UTC

[Impala-ASF-CR] IMPALA-6536: Disable HMS stats collection in createTable().

Alex Behm has uploaded this change for review. ( http://gerrit.cloudera.org:8080/10176


Change subject: IMPALA-6536: Disable HMS stats collection in createTable().
......................................................................

IMPALA-6536: Disable HMS stats collection in createTable().

Sets the DO_NOT_UPDATE_STATS table property to true before
calling the HMS create_table() RPC to avoid the expensive
file listing on the HMS side to compute simple stats.
Those simple stats like file count and file sizes are not
useful to Impala without also running a full COMPUTE STATS.

The expensive file listing can be especially bad for
file systems like S3.

This patch relies on a recent fix in Hive: HIVE-18743

Testing:
- ran test_ddl.py locally
- core/hdfs run passed

No new tests are added:
- Our mini cluster is set up with hive.stats.autogather=false,
  which already prevents the HMS-side stats computation
- Hive has testing as part of HIVE-18743

Change-Id: I2d37f3752106f90e5c0ab2e8c0f5bcca98d6d172
---
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M tests/metadata/test_ddl.py
M tests/metadata/test_show_create_table.py
3 files changed, 12 insertions(+), 1 deletion(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/76/10176/1
-- 
To view, visit http://gerrit.cloudera.org:8080/10176
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I2d37f3752106f90e5c0ab2e8c0f5bcca98d6d172
Gerrit-Change-Number: 10176
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Behm <al...@cloudera.com>

[Impala-ASF-CR] IMPALA-6536: Disable HMS stats collection in createTable().

Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has abandoned this change. ( http://gerrit.cloudera.org:8080/10176 )

Change subject: IMPALA-6536: Disable HMS stats collection in createTable().
......................................................................


Abandoned

I'm going to mark this as abandoned for now so that it doesn't show up in open CR searches. Feel free to revive it if needed.
-- 
To view, visit http://gerrit.cloudera.org:8080/10176
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: abandon
Gerrit-Change-Id: I2d37f3752106f90e5c0ab2e8c0f5bcca98d6d172
Gerrit-Change-Number: 10176
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Dimitris Tsirogiannis <dt...@cloudera.com>