Intial Data Load

[Suresh Chidambaram <ch...@gmail.com>]

Hi Team,

My requirement is that I have to perform an One Time Data Load(intial load)
to the topic, later I have to perform the delta load to the topic. So could
someone guide me how to achieve this requirement in Confluent Kafka/Apache
Kafka?

Thanks
C Suresh

Re: Intial Data Load

[Liam Clarke-Hutchinson <li...@adscale.co.nz>]

Hi Suresh,

So to achieve what you're after, you can use Kafka Connect's JDBC source
with a log compacted topic. It can pick up on new entities (a new primary
id) or updated entities (a mod_time column).

If you want just the deltas, you want to look into Debezium.

Kind regards,

Liam Clarke-Hutchinson

On Sun, 31 May 2020, 7:56 pm Suresh Chidambaram, <ch...@gmail.com>
wrote:

> Hi Liam,
>
> Thank you for the reply. One Time Load from database to Kafka Topic. Kindly
> let me know if more input required.
>
> Thanks
> C Suresh
>
> On Sunday, May 31, 2020, Liam Clarke-Hutchinson <liam.clarke@adscale.co.nz
> >
> wrote:
>
> > Hi Suresh,
> >
> > I'm afraid you're not giving us much to work with. Initial/delta load
> from
> > what?
> >
> > Thanks,
> >
> > Liam Clarke-Hutchinson
> >
> > On Sun, 31 May 2020, 4:50 pm Suresh Chidambaram, <chida.suresh@gmail.com
> >
> > wrote:
> >
> > > Hi Team,
> > >
> > > Could someone help me with my request below?
> > >
> > > Thanks
> > > C Suresh
> > >
> > > On Saturday, May 30, 2020, Suresh Chidambaram <ch...@gmail.com>
> > > wrote:
> > >
> > > > Hi Team,
> > > >
> > > > My requirement is that I have to perform an One Time Data Load(intial
> > > > load) to the topic, later I have to perform the delta load to the
> > topic.
> > > So
> > > > could someone guide me how to achieve this requirement in Confluent
> > > > Kafka/Apache Kafka?
> > > >
> > > > Thanks
> > > > C Suresh
> > > >
> > >
> >
>

Re: Intial Data Load

[Suresh Chidambaram <ch...@gmail.com>]

Hi Liam,

Thank you for the reply. One Time Load from database to Kafka Topic. Kindly
let me know if more input required.

Thanks
C Suresh

On Sunday, May 31, 2020, Liam Clarke-Hutchinson <li...@adscale.co.nz>
wrote:

> Hi Suresh,
>
> I'm afraid you're not giving us much to work with. Initial/delta load from
> what?
>
> Thanks,
>
> Liam Clarke-Hutchinson
>
> On Sun, 31 May 2020, 4:50 pm Suresh Chidambaram, <ch...@gmail.com>
> wrote:
>
> > Hi Team,
> >
> > Could someone help me with my request below?
> >
> > Thanks
> > C Suresh
> >
> > On Saturday, May 30, 2020, Suresh Chidambaram <ch...@gmail.com>
> > wrote:
> >
> > > Hi Team,
> > >
> > > My requirement is that I have to perform an One Time Data Load(intial
> > > load) to the topic, later I have to perform the delta load to the
> topic.
> > So
> > > could someone guide me how to achieve this requirement in Confluent
> > > Kafka/Apache Kafka?
> > >
> > > Thanks
> > > C Suresh
> > >
> >
>

Re: Intial Data Load

[Liam Clarke-Hutchinson <li...@adscale.co.nz>]

Hi Suresh,

I'm afraid you're not giving us much to work with. Initial/delta load from
what?

Thanks,

Liam Clarke-Hutchinson

On Sun, 31 May 2020, 4:50 pm Suresh Chidambaram, <ch...@gmail.com>
wrote:

> Hi Team,
>
> Could someone help me with my request below?
>
> Thanks
> C Suresh
>
> On Saturday, May 30, 2020, Suresh Chidambaram <ch...@gmail.com>
> wrote:
>
> > Hi Team,
> >
> > My requirement is that I have to perform an One Time Data Load(intial
> > load) to the topic, later I have to perform the delta load to the topic.
> So
> > could someone guide me how to achieve this requirement in Confluent
> > Kafka/Apache Kafka?
> >
> > Thanks
> > C Suresh
> >
>

Re: Intial Data Load

[Suresh Chidambaram <ch...@gmail.com>]

Hi Team,

Could someone help me with my request below?

Thanks
C Suresh

On Saturday, May 30, 2020, Suresh Chidambaram <ch...@gmail.com>
wrote:

> Hi Team,
>
> My requirement is that I have to perform an One Time Data Load(intial
> load) to the topic, later I have to perform the delta load to the topic. So
> could someone guide me how to achieve this requirement in Confluent
> Kafka/Apache Kafka?
>
> Thanks
> C Suresh
>