You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Tim Armstrong (Code Review)" <ge...@cloudera.org> on 2018/06/15 21:55:48 UTC
[Impala-ASF-CR] IMPALA-7179: allow multiple scratch dirs per device=true by default
Tim Armstrong has uploaded this change for review. ( http://gerrit.cloudera.org:8080/10736
Change subject: IMPALA-7179: allow_multiple_scratch_dirs_per_device=true by default
......................................................................
IMPALA-7179: allow_multiple_scratch_dirs_per_device=true by default
The previous default was often confusing to users of Impala. It is
simpler to do exactly what is asked instead of trying to fix bad
configurations automatically.
Testing:
Ran core tests.
Change-Id: I23394c9949ae4cd0a21d7bb25551371b3198e76c
---
M tests/custom_cluster/test_scratch_disk.py
1 file changed, 2 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/36/10736/1
--
To view, visit http://gerrit.cloudera.org:8080/10736
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I23394c9949ae4cd0a21d7bb25551371b3198e76c
Gerrit-Change-Number: 10736
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
[Impala-ASF-CR] IMPALA-7179: allow multiple scratch dirs per device=true by default
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/10736 )
Change subject: IMPALA-7179: allow_multiple_scratch_dirs_per_device=true by default
......................................................................
Patch Set 2:
I couldn't think of anything that surprising - worst case performance of spilling might get worse because we don't load-balance across disks as well.
--
To view, visit http://gerrit.cloudera.org:8080/10736
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I23394c9949ae4cd0a21d7bb25551371b3198e76c
Gerrit-Change-Number: 10736
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: anujphadke <ap...@cloudera.com>
Gerrit-Comment-Date: Fri, 15 Jun 2018 23:41:34 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7179: allow multiple scratch dirs per device=true by default
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10736 )
Change subject: IMPALA-7179: allow_multiple_scratch_dirs_per_device=true by default
......................................................................
Patch Set 2: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/10736
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I23394c9949ae4cd0a21d7bb25551371b3198e76c
Gerrit-Change-Number: 10736
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: anujphadke <ap...@cloudera.com>
Gerrit-Comment-Date: Sat, 16 Jun 2018 07:51:41 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7179: allow multiple scratch dirs per device=true by default
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10736 )
Change subject: IMPALA-7179: allow_multiple_scratch_dirs_per_device=true by default
......................................................................
Patch Set 2:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2688/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/10736
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I23394c9949ae4cd0a21d7bb25551371b3198e76c
Gerrit-Change-Number: 10736
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: anujphadke <ap...@cloudera.com>
Gerrit-Comment-Date: Sat, 16 Jun 2018 04:31:30 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7179: allow multiple scratch dirs per device=true by default
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/10736 )
Change subject: IMPALA-7179: allow_multiple_scratch_dirs_per_device=true by default
......................................................................
IMPALA-7179: allow_multiple_scratch_dirs_per_device=true by default
The previous default was often confusing to users of Impala. It is
simpler to do exactly what is asked instead of trying to fix bad
configurations automatically.
Testing:
Ran core tests.
Change-Id: I23394c9949ae4cd0a21d7bb25551371b3198e76c
---
M be/src/runtime/tmp-file-mgr.cc
M tests/custom_cluster/test_scratch_disk.py
2 files changed, 3 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/36/10736/2
--
To view, visit http://gerrit.cloudera.org:8080/10736
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I23394c9949ae4cd0a21d7bb25551371b3198e76c
Gerrit-Change-Number: 10736
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
[Impala-ASF-CR] IMPALA-7179: allow multiple scratch dirs per device=true by default
Posted by "Dan Hecht (Code Review)" <ge...@cloudera.org>.
Dan Hecht has posted comments on this change. ( http://gerrit.cloudera.org:8080/10736 )
Change subject: IMPALA-7179: allow_multiple_scratch_dirs_per_device=true by default
......................................................................
Patch Set 2: Code-Review+2
Seems better. Is there any weirdness/surprise that can happen with this change in default that we should document?
--
To view, visit http://gerrit.cloudera.org:8080/10736
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I23394c9949ae4cd0a21d7bb25551371b3198e76c
Gerrit-Change-Number: 10736
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: anujphadke <ap...@cloudera.com>
Gerrit-Comment-Date: Fri, 15 Jun 2018 23:38:37 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7179: allow multiple scratch dirs per device=true by default
Posted by "Dan Hecht (Code Review)" <ge...@cloudera.org>.
Dan Hecht has posted comments on this change. ( http://gerrit.cloudera.org:8080/10736 )
Change subject: IMPALA-7179: allow_multiple_scratch_dirs_per_device=true by default
......................................................................
Patch Set 2:
ok
--
To view, visit http://gerrit.cloudera.org:8080/10736
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I23394c9949ae4cd0a21d7bb25551371b3198e76c
Gerrit-Change-Number: 10736
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: anujphadke <ap...@cloudera.com>
Gerrit-Comment-Date: Fri, 15 Jun 2018 23:47:46 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7179: allow multiple scratch dirs per device=true by default
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10736 )
Change subject: IMPALA-7179: allow_multiple_scratch_dirs_per_device=true by default
......................................................................
IMPALA-7179: allow_multiple_scratch_dirs_per_device=true by default
The previous default was often confusing to users of Impala. It is
simpler to do exactly what is asked instead of trying to fix bad
configurations automatically.
Testing:
Ran core tests.
Change-Id: I23394c9949ae4cd0a21d7bb25551371b3198e76c
Reviewed-on: http://gerrit.cloudera.org:8080/10736
Reviewed-by: anujphadke <ap...@cloudera.com>
Reviewed-by: Dan Hecht <dh...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M be/src/runtime/tmp-file-mgr.cc
M tests/custom_cluster/test_scratch_disk.py
2 files changed, 3 insertions(+), 2 deletions(-)
Approvals:
anujphadke: Looks good to me, but someone else must approve
Dan Hecht: Looks good to me, approved
Impala Public Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/10736
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I23394c9949ae4cd0a21d7bb25551371b3198e76c
Gerrit-Change-Number: 10736
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: anujphadke <ap...@cloudera.com>
[Impala-ASF-CR] IMPALA-7179: allow multiple scratch dirs per device=true by default
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/10736 )
Change subject: IMPALA-7179: allow_multiple_scratch_dirs_per_device=true by default
......................................................................
Patch Set 2:
Anuj pointed out that I'd forgot to include the critical part of the change (some git add mistake) - thanks Anuj!
--
To view, visit http://gerrit.cloudera.org:8080/10736
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I23394c9949ae4cd0a21d7bb25551371b3198e76c
Gerrit-Change-Number: 10736
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: anujphadke <ap...@cloudera.com>
Gerrit-Comment-Date: Fri, 15 Jun 2018 23:26:05 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7179: allow multiple scratch dirs per device=true by default
Posted by "anujphadke (Code Review)" <ge...@cloudera.org>.
anujphadke has posted comments on this change. ( http://gerrit.cloudera.org:8080/10736 )
Change subject: IMPALA-7179: allow_multiple_scratch_dirs_per_device=true by default
......................................................................
Patch Set 2: Code-Review+1
--
To view, visit http://gerrit.cloudera.org:8080/10736
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I23394c9949ae4cd0a21d7bb25551371b3198e76c
Gerrit-Change-Number: 10736
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: anujphadke <ap...@cloudera.com>
Gerrit-Comment-Date: Fri, 15 Jun 2018 23:25:05 +0000
Gerrit-HasComments: No