[impala] branch master updated: IMPALA-11999: Ensure iceberg catalog location has correct filesystem prefix for Ozone

[as...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

asherman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git


The following commit(s) were added to refs/heads/master by this push:
     new a453a62d7 IMPALA-11999: Ensure iceberg catalog location has correct filesystem prefix for Ozone
a453a62d7 is described below

commit a453a62d7c6f2deecc3272f261cf2494978ed415
Author: Andrew Sherman <as...@cloudera.com>
AuthorDate: Tue Mar 14 13:58:16 2023 -0700

    IMPALA-11999: Ensure iceberg catalog location has correct filesystem prefix for Ozone
    
    Some iceberg tests run using different forms of iceberg catalog. One of
    these specifies iceberg.catalog_location as one of the TBLPROPERTIES
    when a table is created. This location was hard-coded to a path
    starting with "/test-warehouse", which results in an illegal Ozone
    path. Fix this by wrapping the location with get_fs_path(), which gives
    a legal Ozone path.
    
    TESTING
    - ran end to end tests on an Ozone filesystem.
    
    Change-Id: Iabafb9cb778a923ffe084dd3e61053b2aa8a6b1f
    Reviewed-on: http://gerrit.cloudera.org:8080/19629
    Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
    Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
 tests/util/iceberg_util.py | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/tests/util/iceberg_util.py b/tests/util/iceberg_util.py
index 21703acfd..a97aa5b86 100644
--- a/tests/util/iceberg_util.py
+++ b/tests/util/iceberg_util.py
@@ -17,6 +17,8 @@
 from __future__ import absolute_import, division, print_function
 import datetime
 
+from tests.util.filesystem_utils import get_fs_path
+
 
 class Snapshot(object):
   """Encapsulate an Iceberg Snapshot"""
@@ -123,9 +125,10 @@ class IcebergCatalogs:
   def get_iceberg_catalog_properties(self):
     """Return a list containing TBLPROPERTIES corresponding to various iceberg catalogs.
      The TBLPROPERTIES can be used to create tables."""
+    ice_cat_location = "/test-warehouse/{0}/hadoop_catalog_test/".format(self.database)
+    ice_cat_location_fs = get_fs_path(ice_cat_location)
     hadoop_catalog = ("'iceberg.catalog'='hadoop.catalog', "
-        + "'iceberg.catalog_location'='/test-warehouse/{0}/hadoop_catalog_test/'".format(
-          self.database))
+                      + "'iceberg.catalog_location'='{0}'".format(ice_cat_location_fs))
     return [
       self.hadoop_tables,
       self.hive_catalog,