You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2021/10/21 17:33:00 UTC
[jira] [Commented] (IMPALA-10972) Adapt Impala's tests to behavior
change in HIVE-24920
[ https://issues.apache.org/jira/browse/IMPALA-10972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17432633#comment-17432633 ]
ASF subversion and git services commented on IMPALA-10972:
----------------------------------------------------------
Commit 57982efc21746f6994c11b623fc3cdd1dbbac8a2 in impala's branch refs/heads/master from Joe McDonnell
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=57982ef ]
IMPALA-10972: Fixup test behavior to work with HIVE-24920
With HIVE-24920, the HMS runs in a mode that prohibits
creating a managed directory where the default location
already exists. Some Impala test helper functions copied
files into a particular location and then created a
table without specifying the location in the create statement.
This is no longer possible.
This changes the helper functions in test/common/file_utils.py
to create the table before pulling files in.
Tests:
- Ran a core job against a Hive with HIVE-24920 and
verified that the failures due to changes in
behavior are gone.
- Ran a core job against the current Hive
Change-Id: Idfe5468a0b9e1025ec7a0ad3cdce4793f35ca7ba
Reviewed-on: http://gerrit.cloudera.org:8080/17956
Tested-by: Impala Public Jenkins <im...@cloudera.com>
Reviewed-by: Vihang Karajgaonkar <vi...@cloudera.com>
> Adapt Impala's tests to behavior change in HIVE-24920
> -----------------------------------------------------
>
> Key: IMPALA-10972
> URL: https://issues.apache.org/jira/browse/IMPALA-10972
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Affects Versions: Impala 4.1.0
> Reporter: Joe McDonnell
> Assignee: Joe McDonnell
> Priority: Blocker
> Labels: broken-build
>
> In HIVE-24920, the behavior changed so that creating a managed table that points to an existing location without specifying the location explicitly will give an error.
> This impacts a lot of our tests that use tests/common/file_utils.py's create_table_from_file() or create_table_and_copy_files(), causing a lot of test issues like this:
> {noformat}
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> E INNER EXCEPTION: <class 'beeswaxd.ttypes.BeeswaxException'>
> E MESSAGE: ImpalaRuntimeException: Error making 'createTable' RPC to Hive Metastore:
> E CAUSED BY: MetaException: Exception determining external table location:Default location is not available for table: hdfs://localhost:20500/test-warehouse/test_error_propagation_race_ccd6c80c.db/bad_magic_number{noformat}
> tests/common/file_utils.py populates a table's directory before using a normal create table without the location specified. This is now prohibited.
> tests/common/file_utils.py behaves this way as an attempt to make s3 more consistent back when it did not have full consistency. See IMPALA-7804: [https://github.com/apache/impala/commit/f1a3c47959e17d301f970b7ea2755067d00ae986]
> This is no longer a concern, so these functions should change back to something compatible with HIVE-24920.
> Marking this as a blocker, because it prevents moving to a new Hive.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org