Pull configuration

[Jay van Zyl <ja...@gmail.com>]

Hi Abhishek,

We are pretty excited about using Gobblin for ingestion projects. I
have a few questions to solve some key pull config requirements.
1. Do you have a config available for CSV/TSV to Parquet and JDBC/SQL
to Parquet?
2. We want to pull the pull config from our meta store dynamically.
Have you done dynamic config before?

Ciao, Jay

Jay van Zyl
ecosystem.ai

Re: Pull configuration

[Abhishek Tiwari <ab...@apache.org>]

Hi Jay,

I think someone in the community has worked on Parquet connector but I
can't find a handy reference.
For dynamic configuration, either Gobblin-as-a-Service or config store
might be useful.

@Sudarshan Vasudevan <su...@linkedin.com> do you have any suggestions
for the two questions?

Regards,
Abhishek

On Sat, May 25, 2019 at 11:47 AM Jay van Zyl <ja...@gmail.com> wrote:

> Hi Abhishek,
>
> We are pretty excited about using Gobblin for ingestion projects. I
> have a few questions to solve some key pull config requirements.
> 1. Do you have a config available for CSV/TSV to Parquet and JDBC/SQL
> to Parquet?
> 2. We want to pull the pull config from our meta store dynamically.
> Have you done dynamic config before?
>
> Ciao, Jay
>
> Jay van Zyl
> ecosystem.ai
>