You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "David Knupp (Code Review)" <ge...@cloudera.org> on 2017/06/01 21:57:14 UTC
[Impala-ASF-CR] IMPALA-5413: Skip test seq writer hive compatibility on remote clusters.
David Knupp has uploaded a new change for review.
http://gerrit.cloudera.org:8080/7046
Change subject: IMPALA-5413: Skip test_seq_writer_hive_compatibility on remote clusters.
......................................................................
IMPALA-5413: Skip test_seq_writer_hive_compatibility on remote clusters.
There's no immediately obvious way to confirm that the user currently
running the test will have the necessary access to the hive DB on a
remote cluster, so this test will be intermittently flaky.
Change-Id: I1cc8824800e4339874b9c4e3a84969baf848d941
---
M tests/query_test/test_compressed_formats.py
1 file changed, 2 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/46/7046/1
--
To view, visit http://gerrit.cloudera.org:8080/7046
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I1cc8824800e4339874b9c4e3a84969baf848d941
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: David Knupp <dk...@cloudera.com>
[Impala-ASF-CR] IMPALA-5413: Add a hive user for test seq writer hive compatibility.
Posted by "David Knupp (Code Review)" <ge...@cloudera.org>.
David Knupp has uploaded a new patch set (#2).
Change subject: IMPALA-5413: Add a hive user for test_seq_writer_hive_compatibility.
......................................................................
IMPALA-5413: Add a hive user for test_seq_writer_hive_compatibility.
This patch includes a change to the framework to permit the passing
of a username to the run_stmt_in_hive() method in the ImpalaTestSuite
class, but retains the same default value as before.
This is to allow a test to issue a 'select count(*) from foo' query
through hive. Hive needs to set up a job to perform this query, and
HDFS write access to do so. In typical cases, the HDFS user is 'hdfs'.
however it may be necessary to change this depending on the cluster.
On a local mini-cluster, the username appears to be irrelevant, so
this won't affect locally run tests.
Tested by running the core set of tests on a local minicluster to
make sure there were no regressions. Also confirmed that the test
in question now passes on a remote physical cluster.
Change-Id: I1cc8824800e4339874b9c4e3a84969baf848d941
---
M tests/common/impala_test_suite.py
M tests/query_test/test_compressed_formats.py
2 files changed, 11 insertions(+), 4 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/46/7046/2
--
To view, visit http://gerrit.cloudera.org:8080/7046
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1cc8824800e4339874b9c4e3a84969baf848d941
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: David Knupp <dk...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: David Knupp <dk...@cloudera.com>
[Impala-ASF-CR] IMPALA-5413: Add a hive user for test seq writer hive compatibility.
Posted by "David Knupp (Code Review)" <ge...@cloudera.org>.
David Knupp has posted comments on this change.
Change subject: IMPALA-5413: Add a hive user for test_seq_writer_hive_compatibility.
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/7046/2/tests/query_test/test_compressed_formats.py
File tests/query_test/test_compressed_formats.py:
Line 182: #
> remove?
Done
--
To view, visit http://gerrit.cloudera.org:8080/7046
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I1cc8824800e4339874b9c4e3a84969baf848d941
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: David Knupp <dk...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: David Knupp <dk...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5413: Add a hive user for test seq writer hive compatibility.
Posted by "David Knupp (Code Review)" <ge...@cloudera.org>.
Hello Alex Behm,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/7046
to look at the new patch set (#3).
Change subject: IMPALA-5413: Add a hive user for test_seq_writer_hive_compatibility.
......................................................................
IMPALA-5413: Add a hive user for test_seq_writer_hive_compatibility.
This patch includes a change to the framework to permit the passing
of a username to the run_stmt_in_hive() method in the ImpalaTestSuite
class, but retains the same default value as before.
This is to allow a test to issue a 'select count(*) from foo' query
through hive. Hive needs to set up a job to perform this query, and
HDFS write access to do so. In typical cases, the HDFS user is 'hdfs'.
however it may be necessary to change this depending on the cluster.
On a local mini-cluster, the username appears to be irrelevant, so
this won't affect locally run tests.
Tested by running the core set of tests on a local minicluster to
make sure there were no regressions. Also confirmed that the test
in question now passes on a remote physical cluster.
Change-Id: I1cc8824800e4339874b9c4e3a84969baf848d941
---
M tests/common/impala_test_suite.py
M tests/query_test/test_compressed_formats.py
2 files changed, 10 insertions(+), 4 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/46/7046/3
--
To view, visit http://gerrit.cloudera.org:8080/7046
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1cc8824800e4339874b9c4e3a84969baf848d941
Gerrit-PatchSet: 3
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: David Knupp <dk...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: David Knupp <dk...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
[Impala-ASF-CR] IMPALA-5413: Skip test seq writer hive compatibility on remote clusters.
Posted by "Alex Behm (Code Review)" <ge...@cloudera.org>.
Alex Behm has posted comments on this change.
Change subject: IMPALA-5413: Skip test_seq_writer_hive_compatibility on remote clusters.
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/7046/1/tests/query_test/test_compressed_formats.py
File tests/query_test/test_compressed_formats.py:
Line 179: output = self.run_stmt_in_hive('select count(*) from %s' % table_name)
For clarification: We cannot really run any statements in Hive on a remote cluster? Why would Hive not be able to access this table?
--
To view, visit http://gerrit.cloudera.org:8080/7046
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I1cc8824800e4339874b9c4e3a84969baf848d941
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: David Knupp <dk...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5413: Skip test seq writer hive compatibility on remote clusters.
Posted by "David Knupp (Code Review)" <ge...@cloudera.org>.
David Knupp has posted comments on this change.
Change subject: IMPALA-5413: Skip test_seq_writer_hive_compatibility on remote clusters.
......................................................................
Patch Set 1:
It might be a bug in Hive or beeline? Beeline is throwing a WRITE access error on select count(*) from table, where impala-shell handles it fine.
--
To view, visit http://gerrit.cloudera.org:8080/7046
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I1cc8824800e4339874b9c4e3a84969baf848d941
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: David Knupp <dk...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: David Knupp <dk...@cloudera.com>
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5413: Add a hive user for test seq writer hive compatibility.
Posted by "David Knupp (Code Review)" <ge...@cloudera.org>.
David Knupp has posted comments on this change.
Change subject: IMPALA-5413: Add a hive user for test_seq_writer_hive_compatibility.
......................................................................
Patch Set 3: Code-Review+2
Removed one blank # comment. Carrying +2 from Alex.
--
To view, visit http://gerrit.cloudera.org:8080/7046
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I1cc8824800e4339874b9c4e3a84969baf848d941
Gerrit-PatchSet: 3
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: David Knupp <dk...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: David Knupp <dk...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5413: Add a hive user for test seq writer hive compatibility.
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change.
Change subject: IMPALA-5413: Add a hive user for test_seq_writer_hive_compatibility.
......................................................................
Patch Set 3: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/7046
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I1cc8824800e4339874b9c4e3a84969baf848d941
Gerrit-PatchSet: 3
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: David Knupp <dk...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: David Knupp <dk...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5413: Add a hive user for test seq writer hive compatibility.
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged.
Change subject: IMPALA-5413: Add a hive user for test_seq_writer_hive_compatibility.
......................................................................
IMPALA-5413: Add a hive user for test_seq_writer_hive_compatibility.
This patch includes a change to the framework to permit the passing
of a username to the run_stmt_in_hive() method in the ImpalaTestSuite
class, but retains the same default value as before.
This is to allow a test to issue a 'select count(*) from foo' query
through hive. Hive needs to set up a job to perform this query, and
HDFS write access to do so. In typical cases, the HDFS user is 'hdfs'.
however it may be necessary to change this depending on the cluster.
On a local mini-cluster, the username appears to be irrelevant, so
this won't affect locally run tests.
Tested by running the core set of tests on a local minicluster to
make sure there were no regressions. Also confirmed that the test
in question now passes on a remote physical cluster.
Change-Id: I1cc8824800e4339874b9c4e3a84969baf848d941
Reviewed-on: http://gerrit.cloudera.org:8080/7046
Reviewed-by: David Knupp <dk...@cloudera.com>
Tested-by: Impala Public Jenkins
---
M tests/common/impala_test_suite.py
M tests/query_test/test_compressed_formats.py
2 files changed, 10 insertions(+), 4 deletions(-)
Approvals:
Impala Public Jenkins: Verified
David Knupp: Looks good to me, approved
--
To view, visit http://gerrit.cloudera.org:8080/7046
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I1cc8824800e4339874b9c4e3a84969baf848d941
Gerrit-PatchSet: 4
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: David Knupp <dk...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: David Knupp <dk...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
[Impala-ASF-CR] IMPALA-5413: Skip test seq writer hive compatibility on remote clusters.
Posted by "Alex Behm (Code Review)" <ge...@cloudera.org>.
Alex Behm has posted comments on this change.
Change subject: IMPALA-5413: Skip test_seq_writer_hive_compatibility on remote clusters.
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/7046/1/tests/query_test/test_compressed_formats.py
File tests/query_test/test_compressed_formats.py:
Line 179: output = self.run_stmt_in_hive('select count(*) from %s' % table_name)
> Hmm, good point. It appears this is not the case -- we do have some other t
So no idea what is happening here?
--
To view, visit http://gerrit.cloudera.org:8080/7046
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I1cc8824800e4339874b9c4e3a84969baf848d941
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: David Knupp <dk...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: David Knupp <dk...@cloudera.com>
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5413: Add a hive user for test seq writer hive compatibility.
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change.
Change subject: IMPALA-5413: Add a hive user for test_seq_writer_hive_compatibility.
......................................................................
Patch Set 3:
Build started: http://jenkins.impala.io:8080/job/gerrit-verify-dryrun/717/
--
To view, visit http://gerrit.cloudera.org:8080/7046
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I1cc8824800e4339874b9c4e3a84969baf848d941
Gerrit-PatchSet: 3
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: David Knupp <dk...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: David Knupp <dk...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5413: Skip test seq writer hive compatibility on remote clusters.
Posted by "David Knupp (Code Review)" <ge...@cloudera.org>.
David Knupp has posted comments on this change.
Change subject: IMPALA-5413: Skip test_seq_writer_hive_compatibility on remote clusters.
......................................................................
Patch Set 1:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/7046/1/tests/query_test/test_compressed_formats.py
File tests/query_test/test_compressed_formats.py:
Line 179: output = self.run_stmt_in_hive('select count(*) from %s' % table_name)
> For clarification: We cannot really run any statements in Hive on a remote
Hmm, good point. It appears this is not the case -- we do have some other tests in other modules that use this method. It must be something else.
Line 179: output = self.run_stmt_in_hive('select count(*) from %s' % table_name)
> Just looked at the JIRA and it's not really clear to me what Hive is doing.
Thanks for looking at this Alex. I see what you mean. I'll try to figure out why this is happening.
--
To view, visit http://gerrit.cloudera.org:8080/7046
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I1cc8824800e4339874b9c4e3a84969baf848d941
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: David Knupp <dk...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: David Knupp <dk...@cloudera.com>
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5413: Add a hive user for test seq writer hive compatibility.
Posted by "Alex Behm (Code Review)" <ge...@cloudera.org>.
Alex Behm has posted comments on this change.
Change subject: IMPALA-5413: Add a hive user for test_seq_writer_hive_compatibility.
......................................................................
Patch Set 2: Code-Review+2
(1 comment)
http://gerrit.cloudera.org:8080/#/c/7046/2/tests/query_test/test_compressed_formats.py
File tests/query_test/test_compressed_formats.py:
Line 182: #
remove?
--
To view, visit http://gerrit.cloudera.org:8080/7046
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I1cc8824800e4339874b9c4e3a84969baf848d941
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: David Knupp <dk...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: David Knupp <dk...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5413: Skip test seq writer hive compatibility on remote clusters.
Posted by "Alex Behm (Code Review)" <ge...@cloudera.org>.
Alex Behm has posted comments on this change.
Change subject: IMPALA-5413: Skip test_seq_writer_hive_compatibility on remote clusters.
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/7046/1/tests/query_test/test_compressed_formats.py
File tests/query_test/test_compressed_formats.py:
Line 179: output = self.run_stmt_in_hive('select count(*) from %s' % table_name)
> For clarification: We cannot really run any statements in Hive on a remote
Just looked at the JIRA and it's not really clear to me what Hive is doing. Why is is requesting WRITE access on the path "/user"? Maybe something else is going wrong here?
--
To view, visit http://gerrit.cloudera.org:8080/7046
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I1cc8824800e4339874b9c4e3a84969baf848d941
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: David Knupp <dk...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-HasComments: Yes