You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@impala.apache.org by "Tim Armstrong (Code Review)" <ge...@cloudera.org> on 2016/06/06 21:38:54 UTC
[Impala-CR](cdh5-2.6.0 5.8.0) IMPALA-3669: test scratch disk fails on S3
Tim Armstrong has uploaded a new change for review.
http://gerrit.cloudera.org:8080/3318
Change subject: IMPALA-3669: test_scratch_disk fails on S3
......................................................................
IMPALA-3669: test_scratch_disk fails on S3
Make the test deterministic by using max_block_mgr_memory instead of
mem_limit, so that the non-deterministic scanner memory usage does not
influence the spilling behaviour of the queries.
Testing:
Ran the test locally to confirm that it succeeded.
Change-Id: I9525a029ac020bb5b8bea210a741c9f9c5ec3c75
---
M tests/custom_cluster/test_scratch_disk.py
1 file changed, 8 insertions(+), 7 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/18/3318/1
--
To view, visit http://gerrit.cloudera.org:8080/3318
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I9525a029ac020bb5b8bea210a741c9f9c5ec3c75
Gerrit-PatchSet: 1
Gerrit-Project: Impala
Gerrit-Branch: cdh5-2.6.0_5.8.0
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
[Impala-CR](cdh5-2.6.0 5.8.0) IMPALA-3669: test scratch disk fails on S3
Posted by "Silvius Rus (Code Review)" <ge...@cloudera.org>.
Silvius Rus has posted comments on this change.
Change subject: IMPALA-3669: test_scratch_disk fails on S3
......................................................................
Patch Set 2: Code-Review+2
Approved. Test-only change.
--
To view, visit http://gerrit.cloudera.org:8080/3318
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I9525a029ac020bb5b8bea210a741c9f9c5ec3c75
Gerrit-PatchSet: 2
Gerrit-Project: Impala
Gerrit-Branch: cdh5-2.6.0_5.8.0
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Michael Brown <mi...@cloudera.com>
Gerrit-Reviewer: Silvius Rus <sr...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-HasComments: No
[Impala-CR](cdh5-2.6.0 5.8.0) IMPALA-3669: test scratch disk fails on S3
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change.
Change subject: IMPALA-3669: test_scratch_disk fails on S3
......................................................................
Patch Set 2:
Added Michael in case you want to use this with your exhaustive test fixes.
--
To view, visit http://gerrit.cloudera.org:8080/3318
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I9525a029ac020bb5b8bea210a741c9f9c5ec3c75
Gerrit-PatchSet: 2
Gerrit-Project: Impala
Gerrit-Branch: cdh5-2.6.0_5.8.0
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Michael Brown <mi...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-HasComments: No
[Impala-CR](cdh5-2.6.0 5.8.0) IMPALA-3669: test scratch disk fails on S3
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has submitted this change and it was merged.
Change subject: IMPALA-3669: test_scratch_disk fails on S3
......................................................................
IMPALA-3669: test_scratch_disk fails on S3
Make the test deterministic by using max_block_mgr_memory instead of
mem_limit, so that the non-deterministic scanner memory usage does not
influence the spilling behaviour of the queries.
Testing:
Ran the test locally to confirm that it succeeded. Also manually
computed the memory requirement. The data size to be sorted is ~220MB,
so with a 64MB block manager limit per node, at least one node must
spill.
Change-Id: I9525a029ac020bb5b8bea210a741c9f9c5ec3c75
Reviewed-on: http://gerrit.cloudera.org:8080/3318
Reviewed-by: Dan Hecht <dh...@cloudera.com>
Reviewed-by: Michael Brown <mi...@cloudera.com>
Reviewed-by: Silvius Rus <sr...@cloudera.com>
Tested-by: Tim Armstrong <ta...@cloudera.com>
---
M tests/custom_cluster/test_scratch_disk.py
1 file changed, 8 insertions(+), 7 deletions(-)
Approvals:
Michael Brown: Looks good to me, but someone else must approve
Silvius Rus: Looks good to me, approved
Tim Armstrong: Verified
Dan Hecht: Looks good to me, approved
--
To view, visit http://gerrit.cloudera.org:8080/3318
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I9525a029ac020bb5b8bea210a741c9f9c5ec3c75
Gerrit-PatchSet: 3
Gerrit-Project: Impala
Gerrit-Branch: cdh5-2.6.0_5.8.0
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Michael Brown <mi...@cloudera.com>
Gerrit-Reviewer: Silvius Rus <sr...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
[Impala-CR](cdh5-2.6.0 5.8.0) IMPALA-3669: test scratch disk fails on S3
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change.
Change subject: IMPALA-3669: test_scratch_disk fails on S3
......................................................................
Patch Set 2:
http://sandbox.jenkins.cloudera.com/job/impala-private-build-and-test/3315/
--
To view, visit http://gerrit.cloudera.org:8080/3318
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I9525a029ac020bb5b8bea210a741c9f9c5ec3c75
Gerrit-PatchSet: 2
Gerrit-Project: Impala
Gerrit-Branch: cdh5-2.6.0_5.8.0
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Michael Brown <mi...@cloudera.com>
Gerrit-Reviewer: Silvius Rus <sr...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-HasComments: No
[Impala-CR](cdh5-2.6.0 5.8.0) IMPALA-3669: test scratch disk fails on S3
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change.
Change subject: IMPALA-3669: test_scratch_disk fails on S3
......................................................................
Patch Set 2: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/3318
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I9525a029ac020bb5b8bea210a741c9f9c5ec3c75
Gerrit-PatchSet: 2
Gerrit-Project: Impala
Gerrit-Branch: cdh5-2.6.0_5.8.0
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Michael Brown <mi...@cloudera.com>
Gerrit-Reviewer: Silvius Rus <sr...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-HasComments: No
[Impala-CR](cdh5-2.6.0 5.8.0) IMPALA-3669: test scratch disk fails on S3
Posted by "Michael Brown (Code Review)" <ge...@cloudera.org>.
Michael Brown has posted comments on this change.
Change subject: IMPALA-3669: test_scratch_disk fails on S3
......................................................................
Patch Set 2: Code-Review+1
If you tested it in S3, or you know it'll work due to the change in what memory is limited, this is fine.
--
To view, visit http://gerrit.cloudera.org:8080/3318
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I9525a029ac020bb5b8bea210a741c9f9c5ec3c75
Gerrit-PatchSet: 2
Gerrit-Project: Impala
Gerrit-Branch: cdh5-2.6.0_5.8.0
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Michael Brown <mi...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-HasComments: No
[Impala-CR](cdh5-2.6.0 5.8.0) IMPALA-3669: test scratch disk fails on S3
Posted by "Dan Hecht (Code Review)" <ge...@cloudera.org>.
Dan Hecht has posted comments on this change.
Change subject: IMPALA-3669: test_scratch_disk fails on S3
......................................................................
Patch Set 2: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/3318
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I9525a029ac020bb5b8bea210a741c9f9c5ec3c75
Gerrit-PatchSet: 2
Gerrit-Project: Impala
Gerrit-Branch: cdh5-2.6.0_5.8.0
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-HasComments: No
[Impala-CR](cdh5-2.6.0 5.8.0) IMPALA-3669: test scratch disk fails on S3
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has uploaded a new patch set (#2).
Change subject: IMPALA-3669: test_scratch_disk fails on S3
......................................................................
IMPALA-3669: test_scratch_disk fails on S3
Make the test deterministic by using max_block_mgr_memory instead of
mem_limit, so that the non-deterministic scanner memory usage does not
influence the spilling behaviour of the queries.
Testing:
Ran the test locally to confirm that it succeeded. Also manually
computed the memory requirement. The data size to be sorted is ~220MB,
so with a 64MB block manager limit per node, at least one node must
spill.
Change-Id: I9525a029ac020bb5b8bea210a741c9f9c5ec3c75
---
M tests/custom_cluster/test_scratch_disk.py
1 file changed, 8 insertions(+), 7 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/18/3318/2
--
To view, visit http://gerrit.cloudera.org:8080/3318
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I9525a029ac020bb5b8bea210a741c9f9c5ec3c75
Gerrit-PatchSet: 2
Gerrit-Project: Impala
Gerrit-Branch: cdh5-2.6.0_5.8.0
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>