You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Madhan Sundararajan Devaki (JIRA)" <ji...@apache.org> on 2015/05/11 10:50:00 UTC
[jira] [Updated] (HDFS-8012) Updatable HAR Filesystem
[ https://issues.apache.org/jira/browse/HDFS-8012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Madhan Sundararajan Devaki updated HDFS-8012:
---------------------------------------------
Description:
Improvement: Updatable HAR Filesystem.
The following operations may be supported additionally.
+ Add new files [ -a filename-uri1 filename-uri2 ... / -a dirname-uri1 dirname-uri2 ...]
+ Remove existing files [ -d filename-uri1 filename-uri2 ... / -d dirname-uri1 dirname-uri2 ...]
+ Update/Replace existing files (Optional) [ -u old-filename-uri new-filename-uri]
This is required in cases where data is stored in AVRO format in HDFS and the corresponding .avsc files are used to create Hive external tables.
This will lead to the small files (.avsc files in this case) problem when there are a large number of tables that need to be loaded into Hive as external tables as is the typical case during a Datawarehouse migration.
was:
Is there a plan to support updatable HAR Filesystem? If so, by when is this expected please?
The following operations may be supported.
+ Add new files [ -a filename-uri1 filename-uri2 ... / -a dirname-uri1 dirname-uri2 ...]
+ Remove existing files [ -d filename-uri1 filename-uri2 ... / -d dirname-uri1 dirname-uri2 ...]
+ Update/Replace existing files (Optional) [ -u old-filename-uri new-filename-uri]
This is required in cases where data is stored in AVRO format in HDFS and the corresponding .avsc files are used to create Hive external tables.
This will lead to the small files (.avsc files in this case) problem when there are a large number of tables that need to be loaded into Hive as external tables.
> Updatable HAR Filesystem
> ------------------------
>
> Key: HDFS-8012
> URL: https://issues.apache.org/jira/browse/HDFS-8012
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode, hdfs-client
> Reporter: Madhan Sundararajan Devaki
> Priority: Critical
>
> Improvement: Updatable HAR Filesystem.
> The following operations may be supported additionally.
> + Add new files [ -a filename-uri1 filename-uri2 ... / -a dirname-uri1 dirname-uri2 ...]
> + Remove existing files [ -d filename-uri1 filename-uri2 ... / -d dirname-uri1 dirname-uri2 ...]
> + Update/Replace existing files (Optional) [ -u old-filename-uri new-filename-uri]
> This is required in cases where data is stored in AVRO format in HDFS and the corresponding .avsc files are used to create Hive external tables.
> This will lead to the small files (.avsc files in this case) problem when there are a large number of tables that need to be loaded into Hive as external tables as is the typical case during a Datawarehouse migration.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)