You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Jark Wu (Jira)" <ji...@apache.org> on 2022/07/07 13:05:00 UTC
[jira] [Closed] (FLINK-27244) Support subdirectories with Hive tables
[ https://issues.apache.org/jira/browse/FLINK-27244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jark Wu closed FLINK-27244.
---------------------------
Fix Version/s: 1.16.0
Assignee: luoyuxia
Resolution: Fixed
Fixed in master: 3d14248c104c649dfce065c8f6f6d177be35f3ef
> Support subdirectories with Hive tables
> ---------------------------------------
>
> Key: FLINK-27244
> URL: https://issues.apache.org/jira/browse/FLINK-27244
> Project: Flink
> Issue Type: Sub-task
> Components: Connectors / Hive
> Reporter: luoyuxia
> Assignee: luoyuxia
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.16.0
>
>
> Hive support to read recursive directory by setting the property 'set mapred.input.dir.recursive=true', and Spark also support [such behavior|[https://stackoverflow.com/questions/42026043/how-to-recursively-read-hadoop-files-from-directory-using-spark]].
> For normal case, it won't happed for reading recursive directory. But it may happen in the following case:
> I have a paritioned table `fact_tz` with partition day/hour
> {code:java}
> CREATE TABLE fact_tz(x int) PARTITIONED BY (ds STRING, hr STRING) {code}
> Then I want to create an external table `fact_daily` refering to `fact_tz`, but with a coarse-grained partition day.
> {code:java}
> create external table fact_daily(x int) PARTITIONED BY (ds STRING) location 'fact_tz_localtion' ;
> ALTER TABLE fact_daily ADD PARTITION (ds='1') location 'fact_tz_localtion/ds=1'{code}
> But it wll throw exception "Not a file: fact_tz_localtion/ds=1" when try to query this table `fact_daily` for it's the first level of the origin partition and is actually a directory .
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)