You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Fredy Wijaya (Code Review)" <ge...@cloudera.org> on 2018/08/13 18:46:25 UTC
[Impala-ASF-CR] IMPALA-7428: Fix flaky test shell command line::test large sql
Fredy Wijaya has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11201
Change subject: IMPALA-7428: Fix flaky test_shell_command_line::test_large_sql
......................................................................
IMPALA-7428: Fix flaky test_shell_command_line::test_large_sql
This patch fixes the flaky test by calculating the time spent in Impala
shell more accurately by substracting the total time with the query
execution time since the test focuses on Impala shell performance and
not the Impala performance in general. This patch also reduces the time
limit from 30 seconds to 20 seconds.
Testing:
- Ran all shell tests
Change-Id: Ic87891f34872da65aac5ce02caf01da1c050efa5
---
M tests/shell/test_shell_commandline.py
1 file changed, 10 insertions(+), 5 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/01/11201/1
--
To view, visit http://gerrit.cloudera.org:8080/11201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic87891f34872da65aac5ce02caf01da1c050efa5
Gerrit-Change-Number: 11201
Gerrit-PatchSet: 1
Gerrit-Owner: Fredy Wijaya <fw...@cloudera.com>
[Impala-ASF-CR] IMPALA-7428: Fix flaky test shell command line::test large sql
Posted by "Fredy Wijaya (Code Review)" <ge...@cloudera.org>.
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/11201 )
Change subject: IMPALA-7428: Fix flaky test_shell_command_line::test_large_sql
......................................................................
Patch Set 3:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/11201/2/tests/shell/test_shell_commandline.py
File tests/shell/test_shell_commandline.py:
http://gerrit.cloudera.org:8080/#/c/11201/2/tests/shell/test_shell_commandline.py@681
PS2, Line 681: non-existent
> non-existent
Done
http://gerrit.cloudera.org:8080/#/c/11201/2/tests/shell/test_shell_commandline.py@681
PS2, Line 681: eglig
> remove "to be " ?
Done
http://gerrit.cloudera.org:8080/#/c/11201/2/tests/shell/test_shell_commandline.py@688
PS2, Line 688: os.write(sql_file, "col_{0} as b{1},\n".format(i, i))
: os.write(sql_file, "col_{0} as c{1}{2}\n".format(
: i, i, "," if i < num_cols - 1 else ""))
: os.write(sql_file, "from non_existence_large_ta
> If you're concerned about speed, do you want to consolidate these writes()
With the rewrite, the test now finishes in 1 second or less :) I don't think the speed matters that much anymore.
--
To view, visit http://gerrit.cloudera.org:8080/11201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic87891f34872da65aac5ce02caf01da1c050efa5
Gerrit-Change-Number: 11201
Gerrit-PatchSet: 3
Gerrit-Owner: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Brown <mi...@cloudera.com>
Gerrit-Comment-Date: Tue, 14 Aug 2018 16:50:10 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7428: Fix flaky test shell command line::test large sql
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11201 )
Change subject: IMPALA-7428: Fix flaky test_shell_command_line::test_large_sql
......................................................................
IMPALA-7428: Fix flaky test_shell_command_line::test_large_sql
This patch fixes the flaky test by rewriting the test to perform a
large query from a non-existent table instead since the test focuses
on Impala shell performance and not Impala performance in general.
This patch also reduces the time limit from 30 seconds to 10 seconds
and increases the number of columns in the query to 10K.
Testing:
- Ran all shell tests
Change-Id: Ic87891f34872da65aac5ce02caf01da1c050efa5
Reviewed-on: http://gerrit.cloudera.org:8080/11201
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M tests/shell/test_shell_commandline.py
1 file changed, 12 insertions(+), 18 deletions(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/11201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ic87891f34872da65aac5ce02caf01da1c050efa5
Gerrit-Change-Number: 11201
Gerrit-PatchSet: 5
Gerrit-Owner: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Brown <mi...@cloudera.com>
[Impala-ASF-CR] IMPALA-7428: Fix flaky test shell command line::test large sql
Posted by "Fredy Wijaya (Code Review)" <ge...@cloudera.org>.
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/11201 )
Change subject: IMPALA-7428: Fix flaky test_shell_command_line::test_large_sql
......................................................................
Patch Set 2: Code-Review+1
(2 comments)
Carry Csaba's +1
http://gerrit.cloudera.org:8080/#/c/11201/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/11201/1//COMMIT_MSG@10
PS1, Line 10: istent table instead since the test focuses
: on Impala shell performance and not I
> type: subtract + I think that "subtracting the query
Done
http://gerrit.cloudera.org:8080/#/c/11201/1/tests/shell/test_shell_commandline.py
File tests/shell/test_shell_commandline.py:
http://gerrit.cloudera.org:8080/#/c/11201/1/tests/shell/test_shell_commandline.py@707
PS1, Line 707:
: def test_default_timezone(self):
: """Test that the default
> A potential way to make this simpler: wouldn't it also work to do an invali
I like your suggestion. Since I no longer need the output, I can actually make pass -q in impala-shell to make the test even faster. I'll update the code.
--
To view, visit http://gerrit.cloudera.org:8080/11201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic87891f34872da65aac5ce02caf01da1c050efa5
Gerrit-Change-Number: 11201
Gerrit-PatchSet: 2
Gerrit-Owner: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Brown <mi...@cloudera.com>
Gerrit-Comment-Date: Tue, 14 Aug 2018 16:31:25 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7428: Fix flaky test shell command line::test large sql
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11201 )
Change subject: IMPALA-7428: Fix flaky test_shell_command_line::test_large_sql
......................................................................
Patch Set 3:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/324/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/11201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic87891f34872da65aac5ce02caf01da1c050efa5
Gerrit-Change-Number: 11201
Gerrit-PatchSet: 3
Gerrit-Owner: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Brown <mi...@cloudera.com>
Gerrit-Comment-Date: Tue, 14 Aug 2018 17:27:44 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7428: Fix flaky test shell command line::test large sql
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11201 )
Change subject: IMPALA-7428: Fix flaky test_shell_command_line::test_large_sql
......................................................................
Patch Set 4: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/11201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic87891f34872da65aac5ce02caf01da1c050efa5
Gerrit-Change-Number: 11201
Gerrit-PatchSet: 4
Gerrit-Owner: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Brown <mi...@cloudera.com>
Gerrit-Comment-Date: Tue, 14 Aug 2018 16:55:42 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7428: Fix flaky test shell command line::test large sql
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/11201 )
Change subject: IMPALA-7428: Fix flaky test_shell_command_line::test_large_sql
......................................................................
Patch Set 1: Code-Review+1
(2 comments)
1 optional comment + 1 type, lgtm otherwise.
http://gerrit.cloudera.org:8080/#/c/11201/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/11201/1//COMMIT_MSG@10
PS1, Line 10: substracting the total time with the query
: execution time since the test focuses
type: subtract + I think that "subtracting the query
execution time from the total time" would be better here.
http://gerrit.cloudera.org:8080/#/c/11201/1/tests/shell/test_shell_commandline.py
File tests/shell/test_shell_commandline.py:
http://gerrit.cloudera.org:8080/#/c/11201/1/tests/shell/test_shell_commandline.py@707
PS1, Line 707: # In this test, we are only interested in the performance of Impala shell and not
: # the performance of Impala query. So, we need to substract the total time with the
: # query execution time.
A potential way to make this simpler: wouldn't it also work to do an invalid query instead e.g. removing the "create" part so that the select will refer a non-existent table?
--
To view, visit http://gerrit.cloudera.org:8080/11201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic87891f34872da65aac5ce02caf01da1c050efa5
Gerrit-Change-Number: 11201
Gerrit-PatchSet: 1
Gerrit-Owner: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Brown <mi...@cloudera.com>
Gerrit-Comment-Date: Tue, 14 Aug 2018 11:51:00 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7428: Fix flaky test shell command line::test large sql
Posted by "Michael Brown (Code Review)" <ge...@cloudera.org>.
Michael Brown has posted comments on this change. ( http://gerrit.cloudera.org:8080/11201 )
Change subject: IMPALA-7428: Fix flaky test_shell_command_line::test_large_sql
......................................................................
Patch Set 3: Code-Review+2
(1 comment)
http://gerrit.cloudera.org:8080/#/c/11201/2/tests/shell/test_shell_commandline.py
File tests/shell/test_shell_commandline.py:
http://gerrit.cloudera.org:8080/#/c/11201/2/tests/shell/test_shell_commandline.py@688
PS2, Line 688: os.write(sql_file, "col_{0} as b{1},\n".format(i, i))
: os.write(sql_file, "col_{0} as c{1}{2}\n".format(
: i, i, "," if i < num_cols - 1 else ""))
: os.write(sql_file, "from non_existence_large_ta
> With the rewrite, the test now finishes in 1 second or less :) I don't thin
Done
--
To view, visit http://gerrit.cloudera.org:8080/11201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic87891f34872da65aac5ce02caf01da1c050efa5
Gerrit-Change-Number: 11201
Gerrit-PatchSet: 3
Gerrit-Owner: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Brown <mi...@cloudera.com>
Gerrit-Comment-Date: Tue, 14 Aug 2018 16:51:47 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7428: Fix flaky test shell command line::test large sql
Posted by "Michael Brown (Code Review)" <ge...@cloudera.org>.
Michael Brown has posted comments on this change. ( http://gerrit.cloudera.org:8080/11201 )
Change subject: IMPALA-7428: Fix flaky test_shell_command_line::test_large_sql
......................................................................
Patch Set 2:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/11201/2/tests/shell/test_shell_commandline.py
File tests/shell/test_shell_commandline.py:
http://gerrit.cloudera.org:8080/#/c/11201/2/tests/shell/test_shell_commandline.py@681
PS2, Line 681: non-existence
non-existent
http://gerrit.cloudera.org:8080/#/c/11201/2/tests/shell/test_shell_commandline.py@681
PS2, Line 681: to be
remove "to be " ?
http://gerrit.cloudera.org:8080/#/c/11201/2/tests/shell/test_shell_commandline.py@688
PS2, Line 688: os.write(sql_file, 'col_{0} as a{1},\n'.format(i, i))
: os.write(sql_file, 'col_{0} as b{1},\n'.format(i, i))
: os.write(sql_file, 'col_{0} as c{1}{2}\n'.format(
: i, i, ',' if i < num_cols - 1 else ''))
If you're concerned about speed, do you want to consolidate these writes() into one?
You can use multiline python strings with triple quotes and textwrap.dedent() to generate the string without leading spaces, if you wish. You can also use keyword-based formatting for readability, like "{i}".format(i=i).
--
To view, visit http://gerrit.cloudera.org:8080/11201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic87891f34872da65aac5ce02caf01da1c050efa5
Gerrit-Change-Number: 11201
Gerrit-PatchSet: 2
Gerrit-Owner: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Brown <mi...@cloudera.com>
Gerrit-Comment-Date: Tue, 14 Aug 2018 16:40:25 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7428: Fix flaky test shell command line::test large sql
Posted by "Fredy Wijaya (Code Review)" <ge...@cloudera.org>.
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/11201 )
Change subject: IMPALA-7428: Fix flaky test_shell_command_line::test_large_sql
......................................................................
Patch Set 4:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/11201/4/tests/shell/test_shell_commandline.py
File tests/shell/test_shell_commandline.py:
http://gerrit.cloudera.org:8080/#/c/11201/4/tests/shell/test_shell_commandline.py@682
PS4, Line 682: sql_file, sql_path = tempfile.mkstemp()
> The file will not be deleted if you use mkstemp, see https://docs.python.or
The file is deleted in L705, but I'll keep that in mind in the future.
--
To view, visit http://gerrit.cloudera.org:8080/11201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic87891f34872da65aac5ce02caf01da1c050efa5
Gerrit-Change-Number: 11201
Gerrit-PatchSet: 4
Gerrit-Owner: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Brown <mi...@cloudera.com>
Gerrit-Comment-Date: Tue, 14 Aug 2018 18:57:26 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7428: Fix flaky test shell command line::test large sql
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11201 )
Change subject: IMPALA-7428: Fix flaky test_shell_command_line::test_large_sql
......................................................................
Patch Set 4:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2997/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/11201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic87891f34872da65aac5ce02caf01da1c050efa5
Gerrit-Change-Number: 11201
Gerrit-PatchSet: 4
Gerrit-Owner: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Brown <mi...@cloudera.com>
Gerrit-Comment-Date: Tue, 14 Aug 2018 16:55:43 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7428: Fix flaky test shell command line::test large sql
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11201 )
Change subject: IMPALA-7428: Fix flaky test_shell_command_line::test_large_sql
......................................................................
Patch Set 2:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/321/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/11201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic87891f34872da65aac5ce02caf01da1c050efa5
Gerrit-Change-Number: 11201
Gerrit-PatchSet: 2
Gerrit-Owner: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Brown <mi...@cloudera.com>
Gerrit-Comment-Date: Tue, 14 Aug 2018 17:04:42 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7428: Fix flaky test shell command line::test large sql
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11201 )
Change subject: IMPALA-7428: Fix flaky test_shell_command_line::test_large_sql
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/309/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/11201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic87891f34872da65aac5ce02caf01da1c050efa5
Gerrit-Change-Number: 11201
Gerrit-PatchSet: 1
Gerrit-Owner: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Brown <mi...@cloudera.com>
Gerrit-Comment-Date: Mon, 13 Aug 2018 19:19:35 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7428: Fix flaky test shell command line::test large sql
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/11201 )
Change subject: IMPALA-7428: Fix flaky test_shell_command_line::test_large_sql
......................................................................
Patch Set 4:
(1 comment)
See comment, would be +2 otherwise.
http://gerrit.cloudera.org:8080/#/c/11201/4/tests/shell/test_shell_commandline.py
File tests/shell/test_shell_commandline.py:
http://gerrit.cloudera.org:8080/#/c/11201/4/tests/shell/test_shell_commandline.py@682
PS4, Line 682: sql_file, sql_path = tempfile.mkstemp()
The file will not be deleted if you use mkstemp, see https://docs.python.org/2/library/tempfile.html
Please use NamedTemporaryFile to ensure that the file is cleaned up.
--
To view, visit http://gerrit.cloudera.org:8080/11201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic87891f34872da65aac5ce02caf01da1c050efa5
Gerrit-Change-Number: 11201
Gerrit-PatchSet: 4
Gerrit-Owner: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Brown <mi...@cloudera.com>
Gerrit-Comment-Date: Tue, 14 Aug 2018 18:46:11 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7428: Fix flaky test shell command line::test large sql
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/11201 )
Change subject: IMPALA-7428: Fix flaky test_shell_command_line::test_large_sql
......................................................................
Patch Set 4:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/11201/4/tests/shell/test_shell_commandline.py
File tests/shell/test_shell_commandline.py:
http://gerrit.cloudera.org:8080/#/c/11201/4/tests/shell/test_shell_commandline.py@682
PS4, Line 682: sql_file, sql_path = tempfile.mkstemp()
> The file is deleted in L705, but I'll keep that in mind in the future.
Oops, I missed that somehow.
--
To view, visit http://gerrit.cloudera.org:8080/11201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic87891f34872da65aac5ce02caf01da1c050efa5
Gerrit-Change-Number: 11201
Gerrit-PatchSet: 4
Gerrit-Owner: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Brown <mi...@cloudera.com>
Gerrit-Comment-Date: Tue, 14 Aug 2018 19:21:25 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7428: Fix flaky test shell command line::test large sql
Posted by "Fredy Wijaya (Code Review)" <ge...@cloudera.org>.
Fredy Wijaya has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/11201 )
Change subject: IMPALA-7428: Fix flaky test_shell_command_line::test_large_sql
......................................................................
IMPALA-7428: Fix flaky test_shell_command_line::test_large_sql
This patch fixes the flaky test by rewriting the test to perform a
large query from a non-existent table instead since the test focuses
on Impala shell performance and not Impala performance in general.
This patch also reduces the time limit from 30 seconds to 10 seconds
and increases the number of columns in the query to 10K.
Testing:
- Ran all shell tests
Change-Id: Ic87891f34872da65aac5ce02caf01da1c050efa5
---
M tests/shell/test_shell_commandline.py
1 file changed, 11 insertions(+), 16 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/01/11201/2
--
To view, visit http://gerrit.cloudera.org:8080/11201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic87891f34872da65aac5ce02caf01da1c050efa5
Gerrit-Change-Number: 11201
Gerrit-PatchSet: 2
Gerrit-Owner: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Brown <mi...@cloudera.com>
[Impala-ASF-CR] IMPALA-7428: Fix flaky test shell command line::test large sql
Posted by "Fredy Wijaya (Code Review)" <ge...@cloudera.org>.
Fredy Wijaya has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/11201 )
Change subject: IMPALA-7428: Fix flaky test_shell_command_line::test_large_sql
......................................................................
IMPALA-7428: Fix flaky test_shell_command_line::test_large_sql
This patch fixes the flaky test by rewriting the test to perform a
large query from a non-existent table instead since the test focuses
on Impala shell performance and not Impala performance in general.
This patch also reduces the time limit from 30 seconds to 10 seconds
and increases the number of columns in the query to 10K.
Testing:
- Ran all shell tests
Change-Id: Ic87891f34872da65aac5ce02caf01da1c050efa5
---
M tests/shell/test_shell_commandline.py
1 file changed, 12 insertions(+), 18 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/01/11201/3
--
To view, visit http://gerrit.cloudera.org:8080/11201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic87891f34872da65aac5ce02caf01da1c050efa5
Gerrit-Change-Number: 11201
Gerrit-PatchSet: 3
Gerrit-Owner: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Brown <mi...@cloudera.com>
[Impala-ASF-CR] IMPALA-7428: Fix flaky test shell command line::test large sql
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11201 )
Change subject: IMPALA-7428: Fix flaky test_shell_command_line::test_large_sql
......................................................................
Patch Set 4: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/11201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic87891f34872da65aac5ce02caf01da1c050efa5
Gerrit-Change-Number: 11201
Gerrit-PatchSet: 4
Gerrit-Owner: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Brown <mi...@cloudera.com>
Gerrit-Comment-Date: Tue, 14 Aug 2018 20:09:23 +0000
Gerrit-HasComments: No