You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org> on 2019/10/17 13:59:46 UTC
[Impala-ASF-CR] IMPALA-9055: Impala shouldn't set expiration to NEVER for cache directives.
Zoltan Borok-Nagy has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14485
Change subject: IMPALA-9055: Impala shouldn't set expiration to NEVER for cache directives.
......................................................................
IMPALA-9055: Impala shouldn't set expiration to NEVER for cache directives.
In HdfsCachingUtil we set the expiration of cache directives to never.
This works well until the cache pool has max TTL set. Once max TTL is
set Impala will get an exception when it tries to add caching for tables
or partitions.
I changed HdfsCachingUtil to not set the expiration. This way the cache
directive inherits the expiration from the cache pool.
Testing
Added e2e test that creates a table in a cache pool that has max TTL.
Change-Id: I475b92704b19e337b2e62f766e5b978585bf6583
---
M fe/src/main/java/org/apache/impala/util/HdfsCachingUtil.java
M testdata/bin/setup-hdfs-env.sh
M testdata/workloads/functional-query/queries/QueryTest/hdfs-caching.test
M tests/query_test/test_hdfs_caching.py
4 files changed, 19 insertions(+), 9 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/85/14485/1
--
To view, visit http://gerrit.cloudera.org:8080/14485
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I475b92704b19e337b2e62f766e5b978585bf6583
Gerrit-Change-Number: 14485
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-9055: Impala shouldn't set expiration to NEVER for cache directives.
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14485 )
Change subject: IMPALA-9055: Impala shouldn't set expiration to NEVER for cache directives.
......................................................................
Patch Set 1: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/14485
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I475b92704b19e337b2e62f766e5b978585bf6583
Gerrit-Change-Number: 14485
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 17 Oct 2019 18:23:40 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9055: Impala shouldn't set expiration to NEVER for cache directives.
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Zoltan Borok-Nagy has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14485 )
Change subject: IMPALA-9055: Impala shouldn't set expiration to NEVER for cache directives.
......................................................................
IMPALA-9055: Impala shouldn't set expiration to NEVER for cache directives.
In HdfsCachingUtil we set the expiration of cache directives to never.
This works well until the cache pool has max TTL set. Once max TTL is
set Impala will get an exception when it tries to add caching for tables
or partitions.
I changed HdfsCachingUtil to not set the expiration. This way the cache
directive inherits the expiration from the cache pool.
Testing
Added e2e test that creates a table in a cache pool that has max TTL.
Change-Id: I475b92704b19e337b2e62f766e5b978585bf6583
Reviewed-on: http://gerrit.cloudera.org:8080/14485
Reviewed-by: Csaba Ringhofer <cs...@cloudera.com>
Reviewed-by: Tim Armstrong <ta...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M fe/src/main/java/org/apache/impala/util/HdfsCachingUtil.java
M testdata/bin/setup-hdfs-env.sh
M testdata/workloads/functional-query/queries/QueryTest/hdfs-caching.test
M tests/query_test/test_hdfs_caching.py
4 files changed, 19 insertions(+), 9 deletions(-)
Approvals:
Csaba Ringhofer: Looks good to me, but someone else must approve
Tim Armstrong: Looks good to me, approved
Impala Public Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/14485
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I475b92704b19e337b2e62f766e5b978585bf6583
Gerrit-Change-Number: 14485
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-9055: Impala shouldn't set expiration to NEVER for cache directives.
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/14485 )
Change subject: IMPALA-9055: Impala shouldn't set expiration to NEVER for cache directives.
......................................................................
Patch Set 1: Code-Review+1
--
To view, visit http://gerrit.cloudera.org:8080/14485
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I475b92704b19e337b2e62f766e5b978585bf6583
Gerrit-Change-Number: 14485
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Thu, 17 Oct 2019 14:49:01 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9055: Impala shouldn't set expiration to NEVER for cache directives.
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14485 )
Change subject: IMPALA-9055: Impala shouldn't set expiration to NEVER for cache directives.
......................................................................
Patch Set 1:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5101/ DRY_RUN=true
--
To view, visit http://gerrit.cloudera.org:8080/14485
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I475b92704b19e337b2e62f766e5b978585bf6583
Gerrit-Change-Number: 14485
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Thu, 17 Oct 2019 14:02:39 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9055: Impala shouldn't set expiration to NEVER for cache directives.
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14485 )
Change subject: IMPALA-9055: Impala shouldn't set expiration to NEVER for cache directives.
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/4814/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/14485
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I475b92704b19e337b2e62f766e5b978585bf6583
Gerrit-Change-Number: 14485
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Thu, 17 Oct 2019 14:44:29 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9055: Impala shouldn't set expiration to NEVER for cache directives.
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14485 )
Change subject: IMPALA-9055: Impala shouldn't set expiration to NEVER for cache directives.
......................................................................
Patch Set 1: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/14485
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I475b92704b19e337b2e62f766e5b978585bf6583
Gerrit-Change-Number: 14485
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 17 Oct 2019 15:49:15 +0000
Gerrit-HasComments: No