You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Duy Truong <du...@gmail.com> on 2017/08/18 11:13:24 UTC

Global KTable value is null in Kafka Stream left join

Hi everyone,

When using left join, I can't get the value of Global KTable record in
ValueJoiner parameter (3rd parameter). Here is my code:

val userTable: GlobalKTable[String, UserData] =
builder.globalTable(Serdes.String(), userDataSede, userTopic, userDataStore)

val jvnStream: KStream[String, JVNModel] = sourceStream.leftJoin(userTable,
      (eventId: String, dataLog: DataLog) => {
        dataLog.rawData.userId
      },
      (dataLog, userData: UserData) => {
        // userData is null.

      })

What I have to do to resolve this issue?

Thanks
-- 
*Duy Truong*

Re: Global KTable value is null in Kafka Stream left join

Posted by Duy Truong <du...@gmail.com>.
Hi Damian,

I've checked the global table and found that there is no data in the table,
here is my code to check:

val view: ReadOnlyKeyValueStore[String, UserData] =
jvnStream.store("userdata", QueryableStoreTypes.keyValueStore[String,
UserData]())
view.all().foreach((kv) => kv.toString)

And the code to build the table:

val userTable: GlobalKTable[String, UserData] =
builder.globalTable(Serdes.String(), userDataSede, userTopic, userDataStore)

data in kafka topic is serialized in avro format.

Do you have any suggestions?

Thanks

On Fri, Aug 18, 2017 at 6:23 PM, Damian Guy <da...@gmail.com> wrote:

> Hi,
>
> If the userData value is null then that would usually mean that there
> wasn't a record with the provided key in the global table. So you should
> probably check if you have the expected data in the global table and also
> check that your KeyMapper is returning the correct key.
>
> Thanks,
> Damian
>
>
>
> On Fri, 18 Aug 2017 at 12:13 Duy Truong <du...@gmail.com>
> wrote:
>
> > Hi everyone,
> >
> > When using left join, I can't get the value of Global KTable record in
> > ValueJoiner parameter (3rd parameter). Here is my code:
> >
> > val userTable: GlobalKTable[String, UserData] =
> > builder.globalTable(Serdes.String(), userDataSede, userTopic,
> > userDataStore)
> >
> > val jvnStream: KStream[String, JVNModel] = sourceStream.leftJoin(
> userTable,
> >       (eventId: String, dataLog: DataLog) => {
> >         dataLog.rawData.userId
> >       },
> >       (dataLog, userData: UserData) => {
> >         // userData is null.
> >
> >       })
> >
> > What I have to do to resolve this issue?
> >
> > Thanks
> > --
> > *Duy Truong*
> >
>



-- 
*Duy Truong*

Re: Global KTable value is null in Kafka Stream left join

Posted by Damian Guy <da...@gmail.com>.
Hi,

If the userData value is null then that would usually mean that there
wasn't a record with the provided key in the global table. So you should
probably check if you have the expected data in the global table and also
check that your KeyMapper is returning the correct key.

Thanks,
Damian



On Fri, 18 Aug 2017 at 12:13 Duy Truong <du...@gmail.com> wrote:

> Hi everyone,
>
> When using left join, I can't get the value of Global KTable record in
> ValueJoiner parameter (3rd parameter). Here is my code:
>
> val userTable: GlobalKTable[String, UserData] =
> builder.globalTable(Serdes.String(), userDataSede, userTopic,
> userDataStore)
>
> val jvnStream: KStream[String, JVNModel] = sourceStream.leftJoin(userTable,
>       (eventId: String, dataLog: DataLog) => {
>         dataLog.rawData.userId
>       },
>       (dataLog, userData: UserData) => {
>         // userData is null.
>
>       })
>
> What I have to do to resolve this issue?
>
> Thanks
> --
> *Duy Truong*
>