You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@iceberg.apache.org by Pavan Lanka <pl...@apple.com.INVALID> on 2022/11/01 23:57:29 UTC
Re: Configuration of HiveMetastoreClient using Catalog Properties
Hi Ryan,
Wanted a follow up on this one.
Earlier I missed this contribution from Kyle.
https://github.com/apache/iceberg/commit/1b3dbb6f13110eb734488d32e93e0fa8d23e9385
This gives us sufficient options to even provide Hive overrides using the same approach, which makes this patch redundant or at a minimum not in line with this other contribution. I will send out a PR reverting this patch.
Sorry for the double work on this.
Thanks,
Pavan
> On Oct 20, 2022, at 9:48 AM, Ryan Blue <bl...@tabular.io> wrote:
>
> Thanks, Pavan. I just took a look. There's a small nit with formatting but overall I think it looks good.
>
> On Thu, Oct 20, 2022 at 8:21 AM Pavan Lanka <pl...@apple.com.invalid> wrote:
>> Hi,
>>
>> I have raised an issue regarding the means for configuring the HiveMetastoreClient to allow for different properties while connection to different metastore services.
>>
>> https://github.com/apache/iceberg/issues/5988
>>
>> I also have a PR for the same that takes any catalog properties starting with `hive` and uses them in the configuration of the Metastore client.
>>
>> https://github.com/apache/iceberg/pull/5989
>>
>> Would be great to get some eyes on the PR. Please do let me know if I can ease this process in anyway.te
>> Thanks,
>> Pavan
>
>
> --
> Ryan Blue
> Tabular
Re: Configuration of HiveMetastoreClient using Catalog Properties
Posted by Pavan Lanka <pl...@apple.com.INVALID>.
Sorry about the delay. I have raised the following PR to revert the change.

Regards,
Pavan
> On Nov 4, 2022, at 9:33 AM, Ryan Blue <bl...@tabular.io> wrote:
>
> No problem! Thanks for contributing and helping to keep the code clean.
>
> On Tue, Nov 1, 2022 at 4:58 PM Pavan Lanka <pl...@apple.com.invalid> wrote:
>> Hi Ryan,
>>
>> Wanted a follow up on this one.
>>
>> Earlier I missed this contribution from Kyle.
>>
>> SPARK: Allow spark catalogs to have hadoop configuration overrides p…… · apache/iceberg@1b3dbb6
>> github.com
>>
>> <https://github.com/apache/iceberg/commit/1b3dbb6f13110eb734488d32e93e0fa8d23e9385>SPARK: Allow spark catalogs to have hadoop configuration overrides p…… · apache/iceberg@1b3dbb6 <https://github.com/apache/iceberg/commit/1b3dbb6f13110eb734488d32e93e0fa8d23e9385>
>> github.com <https://github.com/apache/iceberg/commit/1b3dbb6f13110eb734488d32e93e0fa8d23e9385> <https://github.com/apache/iceberg/commit/1b3dbb6f13110eb734488d32e93e0fa8d23e9385>
>>
>> This gives us sufficient options to even provide Hive overrides using the same approach, which makes this patch redundant or at a minimum not in line with this other contribution. I will send out a PR reverting this patch.
>>
>> Sorry for the double work on this.
>>
>> Thanks,
>> Pavan
>>
>>> On Oct 20, 2022, at 9:48 AM, Ryan Blue <blue@tabular.io <ma...@tabular.io>> wrote:
>>>
>>> Thanks, Pavan. I just took a look. There's a small nit with formatting but overall I think it looks good.
>>>
>>> On Thu, Oct 20, 2022 at 8:21 AM Pavan Lanka <pl...@apple.com.invalid> wrote:
>>>> Hi,
>>>>
>>>> I have raised an issue regarding the means for configuring the HiveMetastoreClient to allow for different properties while connection to different metastore services.
>>>>
>>>> https://github.com/apache/iceberg/issues/5988
>>>>
>>>> I also have a PR for the same that takes any catalog properties starting with `hive` and uses them in the configuration of the Metastore client.
>>>>
>>>> https://github.com/apache/iceberg/pull/5989
>>>>
>>>> Would be great to get some eyes on the PR. Please do let me know if I can ease this process in anyway.te
>>>> Thanks,
>>>> Pavan
>>>
>>>
>>> --
>>> Ryan Blue
>>> Tabular
>>
>
>
> --
> Ryan Blue
> Tabular
Re: Configuration of HiveMetastoreClient using Catalog Properties
Posted by Ryan Blue <bl...@tabular.io>.
No problem! Thanks for contributing and helping to keep the code clean.
On Tue, Nov 1, 2022 at 4:58 PM Pavan Lanka <pl...@apple.com.invalid> wrote:
> Hi Ryan,
>
> Wanted a follow up on this one.
>
> Earlier I missed this contribution from Kyle.
>
> SPARK: Allow spark catalogs to have hadoop configuration overrides p…… ·
> apache/iceberg@1b3dbb6
> <https://github.com/apache/iceberg/commit/1b3dbb6f13110eb734488d32e93e0fa8d23e9385>
> github.com
> <https://github.com/apache/iceberg/commit/1b3dbb6f13110eb734488d32e93e0fa8d23e9385>
> [image: apple-touch-icon-180x180.png]
> <https://github.com/apache/iceberg/commit/1b3dbb6f13110eb734488d32e93e0fa8d23e9385>
> <https://github.com/apache/iceberg/commit/1b3dbb6f13110eb734488d32e93e0fa8d23e9385>
>
> This gives us sufficient options to even provide Hive overrides using the
> same approach, which makes this patch redundant or at a minimum not in line
> with this other contribution. I will send out a PR reverting this patch.
>
> Sorry for the double work on this.
>
> Thanks,
> Pavan
>
> On Oct 20, 2022, at 9:48 AM, Ryan Blue <bl...@tabular.io> wrote:
>
> Thanks, Pavan. I just took a look. There's a small nit with formatting but
> overall I think it looks good.
>
> On Thu, Oct 20, 2022 at 8:21 AM Pavan Lanka <pl...@apple.com.invalid>
> wrote:
>
>> Hi,
>>
>> I have raised an issue regarding the means for configuring the
>> HiveMetastoreClient to allow for different properties while connection to
>> different metastore services.
>>
>> https://github.com/apache/iceberg/issues/5988
>>
>> I also have a PR for the same that takes any catalog properties starting
>> with `hive` and uses them in the configuration of the Metastore client.
>>
>> https://github.com/apache/iceberg/pull/5989
>>
>> Would be great to get some eyes on the PR. Please do let me know if I can
>> ease this process in anyway.te
>> Thanks,
>> Pavan
>>
>
>
> --
> Ryan Blue
> Tabular
>
>
>
--
Ryan Blue
Tabular