Hive Sync in HUDI

[Kabeer Ahmed <ka...@linuxmail.org>]

All,

Is there an API to do hive sync? The use case is for a pipeline that does update Hive after all the plumbing. I do see that there is a HiveSyncTool() functionality available. But the issue I have with that is with the hardcoded username and password to access Hive. I have currently circumvented it by using Spark.
All I am trying to find out is if there is an API that can handle all the options (like HIVE_ASSUME_DATE_PARTITION_OPT_KEY) without asking for user crendentials to be passed through configuration file or through manual entry.
Thanks
Kabeer.

Re: Hive Sync in HUDI

[Vinoth Chandar <vi...@apache.org>]

Hi Kabeer,

The HiveSyncTool class constructor does take a HiveSyncConfig - which is
what the DeltaStreamer or DataSource use to configure username etc anyway..
So you could try instantiating that directly?

Thanks
Vinoth

On Wed, Apr 10, 2019 at 5:50 AM Kabeer Ahmed <ka...@linuxmail.org> wrote:

> All,
>
> Is there an API to do hive sync? The use case is for a pipeline that does
> update Hive after all the plumbing. I do see that there is a HiveSyncTool()
> functionality available. But the issue I have with that is with the
> hardcoded username and password to access Hive. I have currently
> circumvented it by using Spark.
> All I am trying to find out is if there is an API that can handle all the
> options (like HIVE_ASSUME_DATE_PARTITION_OPT_KEY) without asking for user
> crendentials to be passed through configuration file or through manual
> entry.
> Thanks
> Kabeer.
>