Re: Updating COW Table

[selvaraj periyasamy <se...@gmail.com>]

Thanks for all your inputs . I am able to plugin the custom class and do
cell level merge for my usecase.

Thanks,
Selva

On Tue, Feb 25, 2020 at 5:50 PM leesf <le...@gmail.com> wrote:

> You would pass it via option, like
> option(DataSourceWriteOptions.PAYLOAD_CLASS_OPT_KEY(),
> EmptyHoodieRecordPayload.class.getName())
>
> selvaraj periyasamy <se...@gmail.com> 于2020年2月26日周三
> 上午2:24写道：
>
> > OverwriteWithLatestAvroPayload is used for Delta Streamer. Is there a way
> > for DataSource Writer?
> >
> > please correct me , if I am wrong.
> >
> > Thanks,
> > Selva
> >
> >
> > On Mon, Feb 24, 2020 at 1:15 PM Gary Li <ya...@gmail.com>
> wrote:
> >
> > > Hi, in this case you need to design your own logic to handle merging.
> > > Please check OverwriteWithLatestAvroPlayload class. You can write your
> > own
> > > one and pass it as DataSourceWriteOptions.PAYLOAD_CLASS_OPT_KEY to
> Hudi.
> > >
> > > On Mon, Feb 24, 2020 at 12:25 PM selvaraj periyasamy <
> > > selvaraj.periyasamy1983@gmail.com> wrote:
> > >
> > > > Hi, I am experimenting Hudi 0.5.0 version for some of the update use
> > > cases.
> > > >
> > > > Our flow is as below
> > > >
> > > > RDBMS -> CDC Log -> Hive -> COW table.
> > > >
> > > > CCDC log for update would have the value only for the primary key
> > > columns +
> > > > updated columns. Remaining column values are null. While upserting
> > values
> > > > on CCOW table, would need to update only the column values , which
> are
> > > > updated and retain the values for other columns. When I tested, Hudi
> > > > updates remaining column values as null since log has null values.
> > > >
> > > > Is there a way to merge rows for columns which are having values
> during
> > > > update?
> > > >
> > > > Thanks,
> > > > Selva
> > > >
> > >
> >
>