You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Tim Armstrong (Code Review)" <ge...@cloudera.org> on 2018/04/25 20:53:15 UTC
[Impala-ASF-CR] IMPALA-6906: disable test that depends on memory estimates on S3
Tim Armstrong has uploaded this change for review. ( http://gerrit.cloudera.org:8080/10207
Change subject: IMPALA-6906: disable test that depends on memory estimates on S3
......................................................................
IMPALA-6906: disable test that depends on memory estimates on S3
S3 divides up scan ranges into synthetic blocks smaller than the
equivalent HDFS blocks, which in turn affects the memory estimate
calculation, so the test that was tuned for HDFS does not work
in the same way as S3.
The test is exercising an admission control code path that is
independent of the filesystem, so we don't gain important coverage by
running this on S3.
ADLS and Isilon can have similar block size issues, so skip those too.
Change-Id: Ida763a402203286c02ad3cbcbed5336c70abef7c
---
M tests/custom_cluster/test_admission_controller.py
1 file changed, 9 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/07/10207/1
--
To view, visit http://gerrit.cloudera.org:8080/10207
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ida763a402203286c02ad3cbcbed5336c70abef7c
Gerrit-Change-Number: 10207
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
[Impala-ASF-CR] IMPALA-6906: disable test that depends on memory estimates on S3
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10207 )
Change subject: IMPALA-6906: disable test that depends on memory estimates on S3
......................................................................
IMPALA-6906: disable test that depends on memory estimates on S3
S3 divides up scan ranges into synthetic blocks smaller than the
equivalent HDFS blocks, which in turn affects the memory estimate
calculation, so the test that was tuned for HDFS does not work
in the same way as S3.
The test is exercising an admission control code path that is
independent of the filesystem, so we don't gain important coverage by
running this on S3.
ADLS can have similar block size issues, so skip that too.
Change-Id: Ida763a402203286c02ad3cbcbed5336c70abef7c
Reviewed-on: http://gerrit.cloudera.org:8080/10207
Reviewed-by: Dan Hecht <dh...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M tests/custom_cluster/test_admission_controller.py
1 file changed, 7 insertions(+), 1 deletion(-)
Approvals:
Dan Hecht: Looks good to me, approved
Impala Public Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/10207
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ida763a402203286c02ad3cbcbed5336c70abef7c
Gerrit-Change-Number: 10207
Gerrit-PatchSet: 4
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
[Impala-ASF-CR] IMPALA-6906: disable test that depends on memory estimates on S3
Posted by "Dan Hecht (Code Review)" <ge...@cloudera.org>.
Dan Hecht has posted comments on this change. ( http://gerrit.cloudera.org:8080/10207 )
Change subject: IMPALA-6906: disable test that depends on memory estimates on S3
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/10207/1/tests/custom_cluster/test_admission_controller.py
File tests/custom_cluster/test_admission_controller.py:
http://gerrit.cloudera.org:8080/#/c/10207/1/tests/custom_cluster/test_admission_controller.py@397
PS1, Line 397: filesystems with different (synthetic) block sizes."""
Isilon is HDFS. It should respect block size.
--
To view, visit http://gerrit.cloudera.org:8080/10207
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ida763a402203286c02ad3cbcbed5336c70abef7c
Gerrit-Change-Number: 10207
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Comment-Date: Wed, 25 Apr 2018 22:28:29 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6906: disable test that depends on memory estimates on S3
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/10207 )
Change subject: IMPALA-6906: disable test that depends on memory estimates on S3
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/10207/1/tests/custom_cluster/test_admission_controller.py
File tests/custom_cluster/test_admission_controller.py:
http://gerrit.cloudera.org:8080/#/c/10207/1/tests/custom_cluster/test_admission_controller.py@397
PS1, Line 397: filesystems with different (synthetic) block sizes."""
> Isilon is HDFS. It should respect block size.
Done
--
To view, visit http://gerrit.cloudera.org:8080/10207
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ida763a402203286c02ad3cbcbed5336c70abef7c
Gerrit-Change-Number: 10207
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Wed, 25 Apr 2018 22:31:15 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6906: disable test that depends on memory estimates on S3
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10207 )
Change subject: IMPALA-6906: disable test that depends on memory estimates on S3
......................................................................
Patch Set 3: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/10207
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ida763a402203286c02ad3cbcbed5336c70abef7c
Gerrit-Change-Number: 10207
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 26 Apr 2018 02:59:26 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6906: disable test that depends on memory estimates on S3
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Hello Bikramjeet Vig, Dan Hecht,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/10207
to look at the new patch set (#3).
Change subject: IMPALA-6906: disable test that depends on memory estimates on S3
......................................................................
IMPALA-6906: disable test that depends on memory estimates on S3
S3 divides up scan ranges into synthetic blocks smaller than the
equivalent HDFS blocks, which in turn affects the memory estimate
calculation, so the test that was tuned for HDFS does not work
in the same way as S3.
The test is exercising an admission control code path that is
independent of the filesystem, so we don't gain important coverage by
running this on S3.
ADLS can have similar block size issues, so skip that too.
Change-Id: Ida763a402203286c02ad3cbcbed5336c70abef7c
---
M tests/custom_cluster/test_admission_controller.py
1 file changed, 7 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/07/10207/3
--
To view, visit http://gerrit.cloudera.org:8080/10207
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ida763a402203286c02ad3cbcbed5336c70abef7c
Gerrit-Change-Number: 10207
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
[Impala-ASF-CR] IMPALA-6906: disable test that depends on memory estimates on S3
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Hello Bikramjeet Vig, Dan Hecht,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/10207
to look at the new patch set (#2).
Change subject: IMPALA-6906: disable test that depends on memory estimates on S3
......................................................................
IMPALA-6906: disable test that depends on memory estimates on S3
S3 divides up scan ranges into synthetic blocks smaller than the
equivalent HDFS blocks, which in turn affects the memory estimate
calculation, so the test that was tuned for HDFS does not work
in the same way as S3.
The test is exercising an admission control code path that is
independent of the filesystem, so we don't gain important coverage by
running this on S3.
ADLS and Isilon can have similar block size issues, so skip those too.
Change-Id: Ida763a402203286c02ad3cbcbed5336c70abef7c
---
M tests/custom_cluster/test_admission_controller.py
1 file changed, 7 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/07/10207/2
--
To view, visit http://gerrit.cloudera.org:8080/10207
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ida763a402203286c02ad3cbcbed5336c70abef7c
Gerrit-Change-Number: 10207
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
[Impala-ASF-CR] IMPALA-6906: disable test that depends on memory estimates on S3
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10207 )
Change subject: IMPALA-6906: disable test that depends on memory estimates on S3
......................................................................
Patch Set 3:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2365/
--
To view, visit http://gerrit.cloudera.org:8080/10207
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ida763a402203286c02ad3cbcbed5336c70abef7c
Gerrit-Change-Number: 10207
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Wed, 25 Apr 2018 22:47:07 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6906: disable test that depends on memory estimates on S3
Posted by "Dan Hecht (Code Review)" <ge...@cloudera.org>.
Dan Hecht has posted comments on this change. ( http://gerrit.cloudera.org:8080/10207 )
Change subject: IMPALA-6906: disable test that depends on memory estimates on S3
......................................................................
Patch Set 3: Code-Review+2
Later we could disable on Isilon if it becomes problematic too but then we should put the real reason for that.
--
To view, visit http://gerrit.cloudera.org:8080/10207
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ida763a402203286c02ad3cbcbed5336c70abef7c
Gerrit-Change-Number: 10207
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Wed, 25 Apr 2018 22:33:14 +0000
Gerrit-HasComments: No