You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by "Aditya Gupta (Jira)" <ji...@apache.org> on 2025/06/18 12:16:00 UTC
[jira] [Resolved] (ATLAS-5034) Entities not created in Atlas when hive table created on top a OFS/O3FS bucket/volume
[ https://issues.apache.org/jira/browse/ATLAS-5034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aditya Gupta resolved ATLAS-5034.
---------------------------------
Resolution: Fixed
> Entities not created in Atlas when hive table created on top a OFS/O3FS bucket/volume
> -------------------------------------------------------------------------------------
>
> Key: ATLAS-5034
> URL: https://issues.apache.org/jira/browse/ATLAS-5034
> Project: Atlas
> Issue Type: Bug
> Reporter: Paresh Devalia
> Assignee: Aditya Gupta
> Priority: Major
> Attachments: Screenshot 2024-08-22 at 10.35.16 AM.png
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Facing issues when trying to create hive tables on top of ozone ofs buckets/volumes
> Steps to reproduce
> {code:java}
> # ozone sh volume create /volume1
> # ozone sh bucket create /volume1/bucket1
> # ozone sh bucket create /volume1/bucket1.bucket1.bucket1
> # beeline
> 0: jdbc:hive2://altas-hive> CREATE EXTERNAL TABLE table_volume(id int) row format delimited fields terminated by ' ' stored as textfile location 'ofs://ozone1724268703/volume1';
> INFO : Compiling command(queryId=hive_20240822174025_98caa1a2-f814-4d83-9f9a-31b157a49a44): CREATE EXTERNAL TABLE table_volume(id int) row format delimited fields terminated by ' ' stored as textfile location 'ofs://ozone1724268703/volume1'
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling command(queryId=hive_20240822174025_98caa1a2-f814-4d83-9f9a-31b157a49a44); Time taken: 0.053 seconds
> INFO : Executing command(queryId=hive_20240822174025_98caa1a2-f814-4d83-9f9a-31b157a49a44): CREATE EXTERNAL TABLE table_volume(id int) row format delimited fields terminated by ' ' stored as textfile location 'ofs://ozone1724268703/volume1'
> INFO : Starting task [Stage-0:DDL] in serial mode
> INFO : Completed executing command(queryId=hive_20240822174025_98caa1a2-f814-4d83-9f9a-31b157a49a44); Time taken: 0.417 seconds
> INFO : OK
> No rows affected (0.544 seconds)
>
> 0: jdbc:hive2://atlas-hive> CREATE EXTERNAL TABLE table_bucket_dotted(id int) row format delimited fields terminated by ' ' stored as textfile location 'ofs://ozone1724268703/volume1/bucket1.bucket1.bucket1';
> INFO : Compiling command(queryId=hive_20240822172811_7868d63e-ff31-462b-bcce-7dd6fe9d49fa): CREATE EXTERNAL TABLE table_bucket_dotted(id int) row format delimited fields terminated by ' ' stored as textfile location 'ofs://ozone1724268703/volume1/bucket1.bucket1.bucket1'
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling command(queryId=hive_20240822172811_7868d63e-ff31-462b-bcce-7dd6fe9d49fa); Time taken: 0.182 seconds
> INFO : Executing command(queryId=hive_20240822172811_7868d63e-ff31-462b-bcce-7dd6fe9d49fa): CREATE EXTERNAL TABLE table_bucket_dotted(id int) row format delimited fields terminated by ' ' stored as textfile location 'ofs://ozone1724268703/volume1/bucket1.bucket1.bucket1'
> INFO : Starting task [Stage-0:DDL] in serial mode
> INFO : Completed executing command(queryId=hive_20240822172811_7868d63e-ff31-462b-bcce-7dd6fe9d49fa); Time taken: 0.465 seconds
> INFO : OK
> No rows affected (1.155 seconds)
>
> 0: jdbc:hive2://atlas-hive> CREATE EXTERNAL TABLE table_key_bucket_dotted(id int) row format delimited fields terminated by ' ' stored as textfile location 'ofs://ozone1724268703/volume1/bucket1.bucket1.bucket1/key2';
> INFO : Compiling command(queryId=hive_20240822173256_3da9c7ff-76b8-4116-8b5b-929428b77421): CREATE EXTERNAL TABLE table_key_bucket_dotted(id int) row format delimited fields terminated by ' ' stored as textfile location 'ofs://ozone1724268703/volume1/bucket1.bucket1.bucket1/key2'
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling command(queryId=hive_20240822173256_3da9c7ff-76b8-4116-8b5b-929428b77421); Time taken: 0.026 seconds
> INFO : Executing command(queryId=hive_20240822173256_3da9c7ff-76b8-4116-8b5b-929428b77421): CREATE EXTERNAL TABLE table_key_bucket_dotted(id int) row format delimited fields terminated by ' ' stored as textfile location 'ofs://ozone1724268703/volume1/bucket1.bucket1.bucket1/key2'
> INFO : Starting task [Stage-0:DDL] in serial mode
> INFO : Completed executing command(queryId=hive_20240822173256_3da9c7ff-76b8-4116-8b5b-929428b77421); Time taken: 0.226 seconds
> INFO : OK
> No rows affected (0.342 seconds)
>
> 0: jdbc:hive2://atlas-hive> show tables;
> INFO : Compiling command(queryId=hive_20240822174219_dc6e9ada-4c30-4e82-84cb-d2c0ec071515): show tables
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Created Hive schema: Schema(fieldSchemas:[FieldSchema(name:tab_name, type:string, comment:from deserializer)], properties:null)
> INFO : Completed compiling command(queryId=hive_20240822174219_dc6e9ada-4c30-4e82-84cb-d2c0ec071515); Time taken: 0.045 seconds
> INFO : Executing command(queryId=hive_20240822174219_dc6e9ada-4c30-4e82-84cb-d2c0ec071515): show tables
> INFO : Starting task [Stage-0:DDL] in serial mode
> INFO : Completed executing command(queryId=hive_20240822174219_dc6e9ada-4c30-4e82-84cb-d2c0ec071515); Time taken: 0.026 seconds
> INFO : OK
> +--------------------------+
> | tab_name |
> +--------------------------+
> | sample_07 |
> | sample_08 |
> | table_bucket_dotted |
> | table_key |
> | table_key_bucket_dotted |
> | table_volume |
> | web_logs |
> +--------------------------+
> In Atlas, key based tables are created properly (eg table_key, table_key_bucket_dotted). However, bucket/volume based tables do not show up in Atlas
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)