[jira] [Commented] (IMPALA-10092) Some tests in custom_cluster/test_kudu.py do not run even they are not explicitly disabled.

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-10092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17219792#comment-17219792 ] 

ASF subversion and git services commented on IMPALA-10092:
----------------------------------------------------------

Commit eda06f41ce004cfc7a35d7d26d93e76d54c7cdb0 in impala's branch refs/heads/master from Fang-Yu Rao
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=eda06f4 ]

IMPALA-9990: Support SET OWNER for Kudu tables

KUDU-3090 adds the support for table ownership and exposes the API's of
setting owner on creating and altering tables, which allows Impala to
also pass to Kudu the new owner of the Kudu table for the ALTER TABLE
SET OWNER statement.

Specifically, based on the API of AlterTableOptions#setOwner(), this
patch stores the ownership information of the Kudu table in the
corresponding instance of AlterTableOptions, which will then be passed
to Kudu via a KuduClient.

Testing:
- Added a FE test in AnalyzeKuduDDLTest.java to verify the statement
  could be correctly analyzed.
- Added an E2E test in kudu_alter.test to verify the statement could be
  correctly executed when the integration between Kudu and HMS is not
  enabled.
- Added an E2E test in kudu_hms_alter.test and verified that the
  statement could be correctly executed when the integration between
  Kudu and HMS is enabled after manually re-enabling
  TestKuduHMSIntegration::test_kudu_alter_table(). Note that this was
  not possible before IMPALA-10092 was resolved due to a bug in the
  class of CustomClusterTestSuite. In addition, we may need to delete
  the Kudu table 'simple' via a Kudu-Python client if the E2E test
  complains that the Kudu table already exists, which may be related to
  IMPALA-8751.
- Manually verified that the views of Kudu server and HMS are consistent
  for a synchronized Kudu table after the ALTER TABLE SET OWNER
  statement even though the Kudu table was once an external and
  non-synchronized table, meaning that the owner from Kudu's perspective
  could be different than that from HMS' perspective. Such a discrepancy
  could be created if we execute the ALTER TABLE SET OWNER statement for
  an external Kudu table with the property of 'external.table.purge'
  being false. The test is performed manually because currently the
  Kudu-Python client adopted in Impala's E2E tests is not up to date so
  that the field of 'owner' cannot be accessed in the E2E tests. On the
  other hand, to verify the owner of a Kudu table from Kudu's
  perspective, we used the latest Kudu-Python client as provided at
  github.com/apache/kudu/tree/master/examples/python/basic-python-example.
- Verified that the patch could pass the exhaustive tests in the DEBUG
  mode.

Change-Id: I29d641efc8db314964bc5ee9828a86d4a44ae95c
Reviewed-on: http://gerrit.cloudera.org:8080/16273
Reviewed-by: Vihang Karajgaonkar <vi...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>


> Some tests in custom_cluster/test_kudu.py do not run even they are not explicitly disabled.
> -------------------------------------------------------------------------------------------
>
>                 Key: IMPALA-10092
>                 URL: https://issues.apache.org/jira/browse/IMPALA-10092
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Infrastructure
>            Reporter: Fang-Yu Rao
>            Assignee: Fang-Yu Rao
>            Priority: Minor
>              Labels: kudu, test
>
> We found that the following tests in https://github.com/apache/impala/blob/master/tests/custom_cluster/test_kudu.py do not run even we remove all the decorators like "{{@SkipIfKudu.no_hybrid_clock}}" or "{{@SkipIfHive3.kudu_hms_notifications_not_supported}}" to skip the tests.
> # {{TestKuduOperations::test_local_tz_conversion_ops}}
> # {{TestKuduClientTimeout::test_impalad_timeout}}
> # {{TestKuduHMSIntegration::test_create_managed_kudu_tables}}
> # {{TestKuduHMSIntegration::test_kudu_alter_table}}
> This may be due to the fact that we add at https://github.com/apache/impala/blob/master/tests/common/custom_cluster_test_suite.py#L78-L80 a constraint that only allows '{{table_format}}' to be "{{text/none}}", i.e., '{{file_format}}' has to be "{{text}}" and '{{compression_codec}}' has to be "{{none}}". It can be verified that by removing this constraint, the tests above could be run after removing all the decorators with '{{table_format}}' being "{{kudu/none}}", which is added according to https://github.com/apache/impala/blob/master/tests/common/test_dimensions.py#L116-L119.
>  
> In this regard, we may need to override the class method of add_test_dimensions() of the classes of {{TestKuduOperations}}, {{TestKuduClientTimeout}}, and {{TestKuduHMSIntegration}} so that once those currently disabled Kudu tests are re-enabled, those tests involving the test vector could be run.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org