You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by pandees waran <pa...@gmail.com> on 2013/07/22 13:47:44 UTC
Wildcard support in specifying file location
Hi,
I am newbie to Hive . While creating external tables, can we use wildcard
to specify file location.
i.e:
STORED AS TEXTFILE LOCATION 's3://root/*/date*/'
Is the above specification valid in hive 0.7.1?
Thanks
Re: Wildcard support in specifying file location
Posted by Navis류승우 <na...@nexr.com>.
As described in HIVE-951, if it's implemented, the grammar might be
implemented something like
LOCATION 'hdfs://data/' 'xyz.*2009????.bz2$';
which is consist of directory (compliant with prev versions) and
optional file regex part.
If HIVE-1662 (file pruning by predicate on FILE_NAME VC ) will be once
committed, I'll revisit the issue.
2013/8/11 Nitin Pawar <ni...@gmail.com>:
> What Lefty said is correct as per my understanding.
>
> By default, hive maps a table with a directory with location parameter
> But then you can alter it to point to a single file. (thats the hack to use
> a file as storage location for hive table).
>
> But it does not support regex based files as storage files for tables yet.
>
> May be someone from hive (dev + designers + architects) will be able to tell
> this if its doable in anyway.
> I just tried it. it fails miserably
>
>
>
>
> On Sun, Aug 11, 2013 at 12:30 AM, Lefty Leverenz <le...@gmail.com>
> wrote:
>>
>> I don't know the answer but my guess is no, you can't use wildcards to
>> specify file locations when creating external tables. Since nobody else has
>> answered I suggest you just try it and see what happens.
>>
>> Or google "hive location wildcard" -- that led me to a related question on
>> stackoverflow
>> (http://stackoverflow.com/questions/14864540/can-i-have-a-hive-external-table-partition-search-recursively)
>> which points to two JIRAs, neither of which is resolved:
>>
>> - HIVE-1083 allow sub-directories for an external table/partition
>> (https://issues.apache.org/jira/browse/HIVE-1083)
>>
>> - HIVE-951 Selectively include EXTERNAL TABLE source files via REGEX
>> (https://issues.apache.org/jira/browse/HIVE-951)
>>
>> If my guess is wrong and you're able to use wildcards, please let me know
>> so I can add that information to the Hive wiki.
>>
>> -- Lefty Leverenz
>>
>>
>> On Mon, Jul 22, 2013 at 7:47 AM, pandees waran <pa...@gmail.com> wrote:
>>>
>>> Hi,
>>>
>>> I am newbie to Hive . While creating external tables, can we use
>>> wildcard to specify file location.
>>> i.e:
>>>
>>> STORED AS TEXTFILE LOCATION 's3://root/*/date*/'
>>>
>>> Is the above specification valid in hive 0.7.1?
>>>
>>> Thanks
>>
>>
>>
>>
>
>
>
> --
> Nitin Pawar
Re: Wildcard support in specifying file location
Posted by Nitin Pawar <ni...@gmail.com>.
What Lefty said is correct as per my understanding.
By default, hive maps a table with a directory with location parameter
But then you can alter it to point to a single file. (thats the hack to use
a file as storage location for hive table).
But it does not support regex based files as storage files for tables yet.
May be someone from hive (dev + designers + architects) will be able to
tell this if its doable in anyway.
I just tried it. it fails miserably
On Sun, Aug 11, 2013 at 12:30 AM, Lefty Leverenz <le...@gmail.com>wrote:
> I don't know the answer but my guess is no, you can't use wildcards to
> specify file locations when creating external tables. Since nobody else
> has answered I suggest you just try it and see what happens.
>
> Or google "hive location wildcard" -- that led me to a related question on
> stackoverflow (
> http://stackoverflow.com/questions/14864540/can-i-have-a-hive-external-table-partition-search-recursively)
> which points to two JIRAs, neither of which is resolved:
>
> - HIVE-1083 allow sub-directories for an external table/partition (
> https://issues.apache.org/jira/browse/HIVE-1083)
>
> - HIVE-951 Selectively include EXTERNAL TABLE source files via REGEX (
> https://issues.apache.org/jira/browse/HIVE-951)
>
> If my guess is wrong and you're able to use wildcards, please let me know
> so I can add that information to the Hive wiki.
>
> -- Lefty Leverenz
>
>
> On Mon, Jul 22, 2013 at 7:47 AM, pandees waran <pa...@gmail.com> wrote:
>
>> Hi,
>>
>> I am newbie to Hive . While creating external tables, can we use
>> wildcard to specify file location.
>> i.e:
>>
>> STORED AS TEXTFILE LOCATION 's3://root/*/date*/'
>>
>> Is the above specification valid in hive 0.7.1?
>>
>> Thanks
>>
>
>
>
>
--
Nitin Pawar
Re: Wildcard support in specifying file location
Posted by Lefty Leverenz <le...@gmail.com>.
I don't know the answer but my guess is no, you can't use wildcards to
specify file locations when creating external tables. Since nobody else
has answered I suggest you just try it and see what happens.
Or google "hive location wildcard" -- that led me to a related question on
stackoverflow (
http://stackoverflow.com/questions/14864540/can-i-have-a-hive-external-table-partition-search-recursively)
which points to two JIRAs, neither of which is resolved:
- HIVE-1083 allow sub-directories for an external table/partition (
https://issues.apache.org/jira/browse/HIVE-1083)
- HIVE-951 Selectively include EXTERNAL TABLE source files via REGEX (
https://issues.apache.org/jira/browse/HIVE-951)
If my guess is wrong and you're able to use wildcards, please let me know
so I can add that information to the Hive wiki.
-- Lefty Leverenz
On Mon, Jul 22, 2013 at 7:47 AM, pandees waran <pa...@gmail.com> wrote:
> Hi,
>
> I am newbie to Hive . While creating external tables, can we use wildcard
> to specify file location.
> i.e:
>
> STORED AS TEXTFILE LOCATION 's3://root/*/date*/'
>
> Is the above specification valid in hive 0.7.1?
>
> Thanks
>