You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Zoltan Haindrich (Jira)" <ji...@apache.org> on 2021/09/28 14:40:00 UTC

[jira] [Commented] (HIVE-25569) Enable table definition over a single file

    [ https://issues.apache.org/jira/browse/HIVE-25569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17421431#comment-17421431 ] 

Zoltan Haindrich commented on HIVE-25569:
-----------------------------------------

Proposed solution: SingleFileSystem

Suppose we have a file in a regular filesystem (hdfs://tmp/f1.txt) - over we want to define a table. To avoid the problems we could get into by setting its parent directory as the table's dir. An sfs wrapped URI could be used: sfs+hdfs://tmp/f1.txt/SINGLEFILE.
Specifying the SINGLEFILE path element instructs this filesystem to show only the f1.txt under that directory.

{code}
$ hdfs dfs -find 'hdfs://localhost:20500/tmp/d1/'
hdfs://localhost:20500/tmp/d1
hdfs://localhost:20500/tmp/d1/f1
hdfs://localhost:20500/tmp/d1/f2
$ hdfs dfs -find 'sfs+hdfs://localhost:20500/tmp/d1/'
sfs+hdfs://localhost:20500/tmp/d1
sfs+hdfs://localhost:20500/tmp/d1/f1
sfs+hdfs://localhost:20500/tmp/d1/f1/SINGLEFILE
sfs+hdfs://localhost:20500/tmp/d1/f1/SINGLEFILE/f1
sfs+hdfs://localhost:20500/tmp/d1/f2
sfs+hdfs://localhost:20500/tmp/d1/f2/SINGLEFILE
sfs+hdfs://localhost:20500/tmp/d1/f2/SINGLEFILE/f2
{code}


> Enable table definition over a single file
> ------------------------------------------
>
>                 Key: HIVE-25569
>                 URL: https://issues.apache.org/jira/browse/HIVE-25569
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Zoltan Haindrich
>            Assignee: Zoltan Haindrich
>            Priority: Major
>
> Suppose there is a directory where multiple files are present - and by a 3rd party database system this is perfectly normal - because its treating a single file as the contents of the table.
> Tables defined in the metastore follow a different principle - tables are considered to be under a directory - and all files under that directory are the contents of that directory.
> To enable seamless migration/evaluation of Hive and other databases using HMS as a metadatabackend the ability to define a table over a single file would be usefull.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)