You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Todd Lipcon (Code Review)" <ge...@cloudera.org> on 2018/07/25 01:32:25 UTC

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel 

Todd Lipcon has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11045


Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................

IMPALA-6335. Allow most shell tests to run in parallel

This adds an IMPALA_HISTFILE environment variable (and --history_file
argument) to the shell which overrides the default location of
~/.impalahistory for the shell history. The shell tests now override
this variable so that they can run in parallel without stomping on each
other's history.

This also fixes a couple flaky test which were previously missing the
"execute_serially" annotation -- that annotation is no longer needed
after this fix.

A couple of the tests still need to be executed serially because they
look at metrics such as the number of executed or running queries, and
those metrics are unstable if other tests run in parallel.

I tested this by running:

  ./bin/impala-py.test tests/shell/test_shell_interactive.py \
    -m 'not execute_serially \
    -n 80 \
    --random

... several times in a row on an 88-core box. Prior to the change,
several would fail each time. Now they pass.

Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
---
M shell/impala_shell.py
M shell/impala_shell_config_defaults.py
M shell/option_parser.py
M tests/custom_cluster/test_shell_interactive_reconnect.py
M tests/shell/test_shell_interactive.py
M tests/shell/util.py
6 files changed, 20 insertions(+), 63 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/11045/1
-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon <to...@apache.org>

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/11045 )

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................


Patch Set 7: Code-Review+2

Forwarding +2


-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 7
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Wed, 08 Aug 2018 03:39:35 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11045 )

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................


Patch Set 3:

No Builds Executed


-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 3
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Mon, 30 Jul 2018 22:34:28 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11045 )

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................


Patch Set 6: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/2902/


-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 6
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Thu, 02 Aug 2018 06:34:49 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Hello Pooja Nilangekar, Fredy Wijaya, Csaba Ringhofer, Tim Armstrong, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/11045

to look at the new patch set (#6).

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................

IMPALA-6335. Allow most shell tests to run in parallel

This adds an IMPALA_HISTFILE environment variable (and --history_file
argument) to the shell which overrides the default location of
~/.impalahistory for the shell history. The shell tests now override
this variable to /dev/null so they don't store history. The tests that
need history use a pytest fixture to use a temporary file for their
history. This allows so that they can run in parallel without stomping
on each other's history.

This also fixes a couple flaky test which were previously missing the
"execute_serially" annotation -- that annotation is no longer needed
after this fix.

A couple of the tests still need to be executed serially because they
look at metrics such as the number of executed or running queries, and
those metrics are unstable if other tests run in parallel.

I tested this by running:

  ./bin/impala-py.test tests/shell/test_shell_interactive.py \
    -m 'not execute_serially' \
    -n 80 \
    --random

... several times in a row on an 88-core box. Prior to the change,
several would fail each time. Now they pass.

Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
---
M shell/impala_shell.py
M shell/impala_shell_config_defaults.py
M shell/option_parser.py
M tests/common/impala_test_suite.py
M tests/custom_cluster/test_shell_interactive_reconnect.py
M tests/shell/test_shell_interactive.py
M tests/shell/util.py
7 files changed, 56 insertions(+), 69 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/11045/6
-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 6
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11045 )

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................


Patch Set 4:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/11045/4/shell/impala_shell_config_defaults.py
File shell/impala_shell_config_defaults.py:

http://gerrit.cloudera.org:8080/#/c/11045/4/shell/impala_shell_config_defaults.py@27
PS4, Line 27: H
flake8: E126 continuation line over-indented for hanging indent


http://gerrit.cloudera.org:8080/#/c/11045/4/tests/shell/test_shell_interactive.py
File tests/shell/test_shell_interactive.py:

http://gerrit.cloudera.org:8080/#/c/11045/4/tests/shell/test_shell_interactive.py@49
PS4, Line 49: def tmp_history_file(request):
flake8: E302 expected 2 blank lines, found 1



-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 4
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Tue, 31 Jul 2018 02:40:04 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/11045 )

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................


Patch Set 2:

(2 comments)

Thanks for the change! I had problems in the past with running multiple shells at the same time and forgetting the history of one them - this will give a tool to avoid this.

http://gerrit.cloudera.org:8080/#/c/11045/2/tests/custom_cluster/test_shell_interactive_reconnect.py
File tests/custom_cluster/test_shell_interactive_reconnect.py:

http://gerrit.cloudera.org:8080/#/c/11045/2/tests/custom_cluster/test_shell_interactive_reconnect.py@35
PS2, Line 35:     p = ImpalaShell("--history_file=/dev/null")
Wouldn't it be better to put this into setup/teardown, so that it cannot be forgotten in new tests? For example https://gerrit.cloudera.org/#/c/10992/ contains a new test, and merging the two changes may lead to side effects.


http://gerrit.cloudera.org:8080/#/c/11045/2/tests/shell/test_shell_interactive.py
File tests/shell/test_shell_interactive.py:

http://gerrit.cloudera.org:8080/#/c/11045/2/tests/shell/test_shell_interactive.py@51
PS2, Line 51:     self.history_file = tempfile.mktemp()
I would prefer to set IMPALA_HISTFILE to /dev/null by default and use NamedTemporaryFile(delete=True) in the few tests that read the history. This could probably make interactive tests a little bit faster.



-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Comment-Date: Wed, 25 Jul 2018 13:20:33 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11045 )

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................


Patch Set 2: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Comment-Date: Wed, 25 Jul 2018 08:45:04 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11045 )

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................


Patch Set 7:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/231/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 7
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Wed, 08 Aug 2018 00:21:43 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11045 )

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................


Patch Set 6:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/155/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 6
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Thu, 02 Aug 2018 03:55:39 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11045 )

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................


Patch Set 6:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2902/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 6
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Thu, 02 Aug 2018 03:23:40 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/11045 )

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................


Patch Set 5: Code-Review+1

Tim: I added you as a reviewer because some of the python style warnings are surprising to me:
Line 27: flake8: E126 continuation line over-indented for hanging indent
Line 49: flake8: E302 expected 2 blank lines, found 1

lgtm otherwise


-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 5
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Tue, 31 Jul 2018 10:13:55 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/11045 )

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/11045/3/shell/option_parser.py
File shell/option_parser.py:

http://gerrit.cloudera.org:8080/#/c/11045/3/shell/option_parser.py@198
PS3, Line 198: history_file
> If I didn't miss something, this option is no longer used in any test. I wo
think you're right, will do.



-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 3
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Mon, 30 Jul 2018 19:50:21 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Hello Pooja Nilangekar, Fredy Wijaya, Csaba Ringhofer, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/11045

to look at the new patch set (#5).

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................

IMPALA-6335. Allow most shell tests to run in parallel

This adds an IMPALA_HISTFILE environment variable (and --history_file
argument) to the shell which overrides the default location of
~/.impalahistory for the shell history. The shell tests now override
this variable to /dev/null so they don't store history. The tests that
need history use a pytest fixture to use a temporary file for their
history. This allows so that they can run in parallel without stomping
on each other's history.

This also fixes a couple flaky test which were previously missing the
"execute_serially" annotation -- that annotation is no longer needed
after this fix.

A couple of the tests still need to be executed serially because they
look at metrics such as the number of executed or running queries, and
those metrics are unstable if other tests run in parallel.

I tested this by running:

  ./bin/impala-py.test tests/shell/test_shell_interactive.py \
    -m 'not execute_serially' \
    -n 80 \
    --random

... several times in a row on an 88-core box. Prior to the change,
several would fail each time. Now they pass.

Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
---
M shell/impala_shell.py
M shell/impala_shell_config_defaults.py
M shell/option_parser.py
M tests/common/impala_test_suite.py
M tests/custom_cluster/test_shell_interactive_reconnect.py
M tests/shell/test_shell_interactive.py
M tests/shell/util.py
7 files changed, 56 insertions(+), 68 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/11045/5
-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 5
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Hello Pooja Nilangekar, Fredy Wijaya, Csaba Ringhofer, Tim Armstrong, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/11045

to look at the new patch set (#7).

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................

IMPALA-6335. Allow most shell tests to run in parallel

This adds an IMPALA_HISTFILE environment variable (and --history_file
argument) to the shell which overrides the default location of
~/.impalahistory for the shell history. The shell tests now override
this variable to /dev/null so they don't store history. The tests that
need history use a pytest fixture to use a temporary file for their
history. This allows so that they can run in parallel without stomping
on each other's history.

This also fixes a couple flaky test which were previously missing the
"execute_serially" annotation -- that annotation is no longer needed
after this fix.

A couple of the tests still need to be executed serially because they
look at metrics such as the number of executed or running queries, and
those metrics are unstable if other tests run in parallel.

I tested this by running:

  ./bin/impala-py.test tests/shell/test_shell_interactive.py \
    -m 'not execute_serially' \
    -n 80 \
    --random

... several times in a row on an 88-core box. Prior to the change,
several would fail each time. Now they pass.

Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
---
M shell/impala_shell.py
M shell/impala_shell_config_defaults.py
M shell/option_parser.py
M tests/common/impala_test_suite.py
M tests/custom_cluster/test_shell_interactive_reconnect.py
M tests/shell/test_shell_interactive.py
M tests/shell/util.py
7 files changed, 57 insertions(+), 70 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/11045/7
-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 7
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/11045 )

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/11045/3/shell/option_parser.py
File shell/option_parser.py:

http://gerrit.cloudera.org:8080/#/c/11045/3/shell/option_parser.py@198
PS3, Line 198: history_file
If I didn't miss something, this option is no longer used in any test. I would prefer to either add a test that exercises it or remove it completely.



-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 3
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Mon, 30 Jul 2018 19:48:52 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11045 )

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/46/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Comment-Date: Wed, 25 Jul 2018 02:08:20 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/11045 )

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................


Patch Set 5: Code-Review+2

As the flake8 warnings seem to be irrelevant and other people are not jumping on the review, I am giving +2.


-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 5
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Wed, 01 Aug 2018 14:41:34 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Hello Pooja Nilangekar, Fredy Wijaya, Csaba Ringhofer, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/11045

to look at the new patch set (#4).

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................

IMPALA-6335. Allow most shell tests to run in parallel

This adds an IMPALA_HISTFILE environment variable (and --history_file
argument) to the shell which overrides the default location of
~/.impalahistory for the shell history. The shell tests now override
this variable to /dev/null so they don't store history. The tests that
need history use a pytest fixture to use a temporary file for their
history. This allows so that they can run in parallel without stomping
on each other's history.

This also fixes a couple flaky test which were previously missing the
"execute_serially" annotation -- that annotation is no longer needed
after this fix.

A couple of the tests still need to be executed serially because they
look at metrics such as the number of executed or running queries, and
those metrics are unstable if other tests run in parallel.

I tested this by running:

  ./bin/impala-py.test tests/shell/test_shell_interactive.py \
    -m 'not execute_serially' \
    -n 80 \
    --random

... several times in a row on an 88-core box. Prior to the change,
several would fail each time. Now they pass.

Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
---
M shell/impala_shell.py
M shell/impala_shell_config_defaults.py
M shell/option_parser.py
M tests/common/impala_test_suite.py
M tests/custom_cluster/test_shell_interactive_reconnect.py
M tests/shell/test_shell_interactive.py
M tests/shell/util.py
7 files changed, 51 insertions(+), 67 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/11045/4
-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 4
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11045 )

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................


Patch Set 1:

Build Started https://jenkins.impala.io/job/gerrit-code-review-checks/46/ 

Running initial code review checks. This is experimental - please report any issues to tarmstrong@cloudera.com or on this JIRA: IMPALA-7317


-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Wed, 25 Jul 2018 01:32:33 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/11045 )

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................


Patch Set 2:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/11045/2/tests/custom_cluster/test_shell_interactive_reconnect.py
File tests/custom_cluster/test_shell_interactive_reconnect.py:

http://gerrit.cloudera.org:8080/#/c/11045/2/tests/custom_cluster/test_shell_interactive_reconnect.py@35
PS2, Line 35:     p = ImpalaShell("--history_file=/dev/null")
> Wouldn't it be better to put this into setup/teardown, so that it cannot be
Will do


http://gerrit.cloudera.org:8080/#/c/11045/2/tests/shell/test_shell_interactive.py
File tests/shell/test_shell_interactive.py:

http://gerrit.cloudera.org:8080/#/c/11045/2/tests/shell/test_shell_interactive.py@51
PS2, Line 51:     self.history_file = tempfile.mktemp()
> I would prefer to set IMPALA_HISTFILE to /dev/null by default and use Named
will do



-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Wed, 25 Jul 2018 16:48:52 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/11045 )

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................


Patch Set 5:

(1 comment)

Fixed some flake8s, though tries to meet half-way between consistency with existing code and making flake8 happy

http://gerrit.cloudera.org:8080/#/c/11045/4/shell/impala_shell_config_defaults.py
File shell/impala_shell_config_defaults.py:

http://gerrit.cloudera.org:8080/#/c/11045/4/shell/impala_shell_config_defaults.py@27
PS4, Line 27: H
> flake8: E126 continuation line over-indented for hanging indent
hmm, do we not do +4 indentation in Python for hanging indents?



-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 5
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Tue, 31 Jul 2018 02:46:24 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11045 )

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................


Patch Set 5: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 5
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Wed, 01 Aug 2018 19:53:33 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/11045 )

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................


Patch Set 5:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/11045/5/tests/shell/test_shell_interactive.py
File tests/shell/test_shell_interactive.py:

http://gerrit.cloudera.org:8080/#/c/11045/5/tests/shell/test_shell_interactive.py@49
PS5, Line 49: def tmp_history_file(request):
> flake8: E302 expected 2 blank lines, found 1
I think Jenkins commented on the wrong revision? Perhaps the script needs to explicitly mention the revision that it looked at when it makes the comment API call to gerrit?



-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 5
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Tue, 31 Jul 2018 16:24:00 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11045 )

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................


Patch Set 4:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/111/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 4
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Tue, 31 Jul 2018 03:12:35 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11045 )

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................


Patch Set 5:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/11045/5/tests/shell/test_shell_interactive.py
File tests/shell/test_shell_interactive.py:

http://gerrit.cloudera.org:8080/#/c/11045/5/tests/shell/test_shell_interactive.py@49
PS5, Line 49: def tmp_history_file(request):
> Hmm I'll take a look
So we do include the patchset number: https://github.com/apache/impala/blob/master/bin/jenkins/critique-gerrit-review.py#L120

I looked at the jenkins job and it looks like the version running against patchset 5 (a50d9842) produced those warnings incorrectly, so I'm going to look at debugging that. Seems like an issue with somehow running flake8 on the wrong commit.



-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 5
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Tue, 31 Jul 2018 16:42:24 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/11045 )

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................


Patch Set 6:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/11045/6/tests/shell/test_shell_interactive.py
File tests/shell/test_shell_interactive.py:

http://gerrit.cloudera.org:8080/#/c/11045/6/tests/shell/test_shell_interactive.py@126
PS6, Line 126: tempfile.
Hint for fixing build: "tempfile." should be removed.



-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 6
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Fri, 03 Aug 2018 12:16:34 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11045 )

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................


Patch Set 2:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2863/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Comment-Date: Wed, 25 Jul 2018 05:22:17 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11045 )

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................

IMPALA-6335. Allow most shell tests to run in parallel

This adds an IMPALA_HISTFILE environment variable (and --history_file
argument) to the shell which overrides the default location of
~/.impalahistory for the shell history. The shell tests now override
this variable to /dev/null so they don't store history. The tests that
need history use a pytest fixture to use a temporary file for their
history. This allows so that they can run in parallel without stomping
on each other's history.

This also fixes a couple flaky test which were previously missing the
"execute_serially" annotation -- that annotation is no longer needed
after this fix.

A couple of the tests still need to be executed serially because they
look at metrics such as the number of executed or running queries, and
those metrics are unstable if other tests run in parallel.

I tested this by running:

  ./bin/impala-py.test tests/shell/test_shell_interactive.py \
    -m 'not execute_serially' \
    -n 80 \
    --random

... several times in a row on an 88-core box. Prior to the change,
several would fail each time. Now they pass.

Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Reviewed-on: http://gerrit.cloudera.org:8080/11045
Tested-by: Impala Public Jenkins <im...@cloudera.com>
Reviewed-by: Todd Lipcon <to...@apache.org>
---
M shell/impala_shell.py
M shell/impala_shell_config_defaults.py
M shell/option_parser.py
M tests/common/impala_test_suite.py
M tests/custom_cluster/test_shell_interactive_reconnect.py
M tests/shell/test_shell_interactive.py
M tests/shell/util.py
7 files changed, 57 insertions(+), 70 deletions(-)

Approvals:
  Impala Public Jenkins: Verified
  Todd Lipcon: Looks good to me, approved

-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 8
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11045 )

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................


Patch Set 5:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/112/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 5
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Tue, 31 Jul 2018 03:19:35 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11045 )

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................


Patch Set 5:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2896/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 5
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Wed, 01 Aug 2018 16:39:57 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11045 )

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................


Patch Set 2:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/47/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Comment-Date: Wed, 25 Jul 2018 02:17:05 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11045 )

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................


Patch Set 5:

(6 comments)

http://gerrit.cloudera.org:8080/#/c/11045/5/shell/impala_shell_config_defaults.py
File shell/impala_shell_config_defaults.py:

http://gerrit.cloudera.org:8080/#/c/11045/5/shell/impala_shell_config_defaults.py@27
PS5, Line 27: H
flake8: E126 continuation line over-indented for hanging indent


http://gerrit.cloudera.org:8080/#/c/11045/5/tests/shell/test_shell_interactive.py
File tests/shell/test_shell_interactive.py:

http://gerrit.cloudera.org:8080/#/c/11045/5/tests/shell/test_shell_interactive.py@49
PS5, Line 49: def tmp_history_file(request):
flake8: E302 expected 2 blank lines, found 1


http://gerrit.cloudera.org:8080/#/c/11045/5/tests/shell/test_shell_interactive.py@66
PS5, Line 66: 
flake8: E701 multiple statements on one line (colon)


http://gerrit.cloudera.org:8080/#/c/11045/5/tests/shell/test_shell_interactive.py@275
PS5, Line 275: 
flake8: E703 statement ends with a semicolon


http://gerrit.cloudera.org:8080/#/c/11045/5/tests/shell/test_shell_interactive.py@276
PS5, Line 276: 
flake8: E703 statement ends with a semicolon


http://gerrit.cloudera.org:8080/#/c/11045/5/tests/shell/test_shell_interactive.py@277
PS5, Line 277: 
flake8: E703 statement ends with a semicolon



-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 5
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Tue, 31 Jul 2018 02:47:06 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11045 )

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................


Patch Set 7:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2951/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 7
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Tue, 07 Aug 2018 23:46:28 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11045 )

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................


Patch Set 5:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/11045/5/tests/shell/test_shell_interactive.py
File tests/shell/test_shell_interactive.py:

http://gerrit.cloudera.org:8080/#/c/11045/5/tests/shell/test_shell_interactive.py@49
PS5, Line 49: def tmp_history_file(request):
> I think Jenkins commented on the wrong revision? Perhaps the script needs t
Hmm I'll take a look



-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 5
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Tue, 31 Jul 2018 16:25:53 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11045 )

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................


Patch Set 7: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 7
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Wed, 08 Aug 2018 03:06:46 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11045 )

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................


Patch Set 2:

Build Started https://jenkins.impala.io/job/gerrit-code-review-checks/47/ 

Running initial code review checks. This is experimental - please report any issues to tarmstrong@cloudera.com or on this JIRA: IMPALA-7317


-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Comment-Date: Wed, 25 Jul 2018 01:34:48 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Hello Pooja Nilangekar, Fredy Wijaya, Csaba Ringhofer, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/11045

to look at the new patch set (#3).

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................

IMPALA-6335. Allow most shell tests to run in parallel

This adds an IMPALA_HISTFILE environment variable (and --history_file
argument) to the shell which overrides the default location of
~/.impalahistory for the shell history. The shell tests now override
this variable to /dev/null so they don't store history. The tests that
need history use a pytest fixture to use a temporary file for their
history. This allows so that they can run in parallel without stomping
on each other's history.

This also fixes a couple flaky test which were previously missing the
"execute_serially" annotation -- that annotation is no longer needed
after this fix.

A couple of the tests still need to be executed serially because they
look at metrics such as the number of executed or running queries, and
those metrics are unstable if other tests run in parallel.

I tested this by running:

  ./bin/impala-py.test tests/shell/test_shell_interactive.py \
    -m 'not execute_serially' \
    -n 80 \
    --random

... several times in a row on an 88-core box. Prior to the change,
several would fail each time. Now they pass.

Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
---
M shell/impala_shell.py
M shell/impala_shell_config_defaults.py
M shell/option_parser.py
M tests/common/impala_test_suite.py
M tests/custom_cluster/test_shell_interactive_reconnect.py
M tests/shell/test_shell_interactive.py
M tests/shell/util.py
7 files changed, 33 insertions(+), 67 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/11045/3
-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 3
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Hello Pooja Nilangekar, Fredy Wijaya, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/11045

to look at the new patch set (#2).

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................

IMPALA-6335. Allow most shell tests to run in parallel

This adds an IMPALA_HISTFILE environment variable (and --history_file
argument) to the shell which overrides the default location of
~/.impalahistory for the shell history. The shell tests now override
this variable so that they can run in parallel without stomping on each
other's history.

This also fixes a couple flaky test which were previously missing the
"execute_serially" annotation -- that annotation is no longer needed
after this fix.

A couple of the tests still need to be executed serially because they
look at metrics such as the number of executed or running queries, and
those metrics are unstable if other tests run in parallel.

I tested this by running:

  ./bin/impala-py.test tests/shell/test_shell_interactive.py \
    -m 'not execute_serially \
    -n 80 \
    --random

... several times in a row on an 88-core box. Prior to the change,
several would fail each time. Now they pass.

Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
---
M shell/impala_shell.py
M shell/impala_shell_config_defaults.py
M shell/option_parser.py
M tests/custom_cluster/test_shell_interactive_reconnect.py
M tests/shell/test_shell_interactive.py
M tests/shell/util.py
6 files changed, 21 insertions(+), 64 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/11045/2
-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>

[Impala-ASF-CR] IMPALA-6335. Allow most shell tests to run in parallel

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/11045 )

Change subject: IMPALA-6335. Allow most shell tests to run in parallel
......................................................................


Patch Set 7:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/11045/6/tests/shell/test_shell_interactive.py
File tests/shell/test_shell_interactive.py:

http://gerrit.cloudera.org:8080/#/c/11045/6/tests/shell/test_shell_interactive.py@126
PS6, Line 126: NamedTemp
> Hint for fixing build: "tempfile." should be removed.
done. Sorry for taking so long to rev the patch, had a lot of other stuff on my plate



-- 
To view, visit http://gerrit.cloudera.org:8080/11045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1da5739276e63a50590dfcb2b050703f8e35fec7
Gerrit-Change-Number: 11045
Gerrit-PatchSet: 7
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <po...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Tue, 07 Aug 2018 23:46:14 +0000
Gerrit-HasComments: Yes