Consume Kafka record - Key and offset from kafka

[KhajaAsmath Mohammed <md...@gmail.com>]

Hi,

Is there a way to get key,timestamp and offset in consume kafka record?

Offset: 380050   Key:
urn:uuid:eb3b516b-9a31-4365-8ccb-e8c009073f63|+|2020-05-05
  Timestamp: 2020-05-05 03:23:54.210 Headers: empty

<https://its-kafka-dev-worker-1.ucsd.edu:9000/dev/topic/LMS_RAW_STREAM_EVENTS_INDEXED_DATA_TAB/messages?partition=0&offset=380050&count=100&format=DEFAULT&msgTypeName=#>

{"ID":"urn:uuid:eb3b516b-9a31-4365-8ccb-e8c009073f63","KSQL_COL_1":2}

Re: Consume Kafka record - Key and offset from kafka

[Bryan Bende <bb...@gmail.com>]

Hello,

Not totally sure if this is what you are asking, but there isn't a way for
them to be added to each record.

The key, timestamp, and offset are all added as flow file attributes, but
the key is only added if you are doing one record per flow file since each
record could have a different key, so there can't be a single key attribute
for the whole flow file when there are N records.

Thanks,

Bryan


On Tue, May 5, 2020 at 12:15 PM KhajaAsmath Mohammed <
mdkhajaasmath@gmail.com> wrote:

> Hi,
>
> Is there a way to get key,timestamp and offset in consume kafka record?
>
> Offset: 380050   Key: urn:uuid:eb3b516b-9a31-4365-8ccb-e8c009073f63|+|2020-05-05
>   Timestamp: 2020-05-05 03:23:54.210 Headers: empty
>
> <https://its-kafka-dev-worker-1.ucsd.edu:9000/dev/topic/LMS_RAW_STREAM_EVENTS_INDEXED_DATA_TAB/messages?partition=0&offset=380050&count=100&format=DEFAULT&msgTypeName=#>
>
> {"ID":"urn:uuid:eb3b516b-9a31-4365-8ccb-e8c009073f63","KSQL_COL_1":2}
>
>