You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Anup Tiwari <an...@gmail.com> on 2020/02/28 12:42:22 UTC
Hive Config for ignoring Glacier Object while querying
Hi Team,
Do we have any config like we have in presto which ignores glacier objects
rather than failing the query ?
*hive.s3.skip-glacier-objects
<https://prestodb.io/docs/current/connector/hive.html> :- Ignore Glacier
objects rather than failing the query. This will skip data that may be
expected to be part of the table or partition. Defaults to false.*
Actually i have a partition tables of which data is on S3. We have archived
some older data to glacier so now if we run query it fails so we have
applied a partition date condition after which query is scanning only those
objects which are on S3 but just wanted to know that do we have any config
for ignoring glacier objects ?
Regards,
Anup Tiwari
Re: Hive Config for ignoring Glacier Object while querying
Posted by David Lavati <dl...@cloudera.com>.
Hi Anup,
I'm not that familiar yet with Hive's S3/Glacier-related capabilities, but
a quick search in both the code base and our jira project returned with
nothing in relation to glacier.
Regards,
David
On Tue, Mar 3, 2020 at 7:48 AM Anup Tiwari <an...@gmail.com> wrote:
> Hi Team,
>
> It will be really helpful if someone can guide me here ?
>
> Regards,
> Anup Tiwari
>
>
> On Fri, Feb 28, 2020 at 6:12 PM Anup Tiwari <an...@gmail.com>
> wrote:
>
>> Hi Team,
>>
>> Do we have any config like we have in presto which ignores glacier
>> objects rather than failing the query ?
>>
>> *hive.s3.skip-glacier-objects
>> <https://prestodb.io/docs/current/connector/hive.html> :- Ignore Glacier
>> objects rather than failing the query. This will skip data that may be
>> expected to be part of the table or partition. Defaults to false.*
>>
>> Actually i have a partition tables of which data is on S3. We have
>> archived some older data to glacier so now if we run query it fails so we
>> have applied a partition date condition after which query is scanning only
>> those objects which are on S3 but just wanted to know that do we have any
>> config for ignoring glacier objects ?
>>
>> Regards,
>> Anup Tiwari
>>
>
Re: Hive Config for ignoring Glacier Object while querying
Posted by Anup Tiwari <an...@gmail.com>.
Hi Team,
It will be really helpful if someone can guide me here ?
Regards,
Anup Tiwari
On Fri, Feb 28, 2020 at 6:12 PM Anup Tiwari <an...@gmail.com> wrote:
> Hi Team,
>
> Do we have any config like we have in presto which ignores glacier objects
> rather than failing the query ?
>
> *hive.s3.skip-glacier-objects
> <https://prestodb.io/docs/current/connector/hive.html> :- Ignore Glacier
> objects rather than failing the query. This will skip data that may be
> expected to be part of the table or partition. Defaults to false.*
>
> Actually i have a partition tables of which data is on S3. We have
> archived some older data to glacier so now if we run query it fails so we
> have applied a partition date condition after which query is scanning only
> those objects which are on S3 but just wanted to know that do we have any
> config for ignoring glacier objects ?
>
> Regards,
> Anup Tiwari
>