You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Parth Gandhi <pa...@excellenceinfonet.com> on 2019/01/10 10:12:33 UTC
Kafka data log timestamp
Hi,
Does kafka record the timestamp of the incoming message in its data log? I
checked one of the partition log and I can see the message without any
timestamp. Also there are few special characters in the message log. IS
that normal?
Here is a sample log: pastebin.com/hStyCW13
Thanks,
Parth Gandhi
DevOps
Disclaimer
The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.
This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd, an innovator in Software as a Service (SaaS) for business. Providing a safer and more useful place for your human generated data. Specializing in; Security, archiving and compliance. To find out more visit the Mimecast website.
Re: Kafka data log timestamp
Posted by Parth Gandhi <pa...@excellenceinfonet.com>.
Thatns Robin. Can you send reference to it? Also can it be used for log
file stored locally (not in DB)?
Thanks,
Parth Gandhi
DevOps
On Thu, Jan 10, 2019 at 9:17 AM Robin Moffatt <ro...@confluent.io> wrote:
> You can use kafkacat to examine the timestamp (and other metadata). Here's
> an example of calling it, and two sample output records:
>
> $ kafkacat -b localhost:9092 -t mysql_users -C -c2 -f '\nKey (%K bytes):
> %k\t\nValue (%S bytes): %s\nTimestamp: %T\tPartition: %p\tOffset: %o\n--\n'
>
> Key (1 bytes): 1
> Value (79 bytes):
> {"uid":1,"name":"Cliff","locale":"en_US","address_city":"St
> Louis","elite":"P"}
> Timestamp: 1520618381093 Partition: 0 Offset: 0
> --
>
> Key (1 bytes): 2
> Value (79 bytes):
> {"uid":2,"name":"Nick","locale":"en_US","address_city":"Palo
> Alto","elite":"G"}
> Timestamp: 1520618381093 Partition: 0 Offset: 1
> --
>
>
> --
>
> Robin Moffatt | Developer Advocate | robin@confluent.io | @rmoff
>
>
> On Thu, 10 Jan 2019 at 10:13, Parth Gandhi <
> parth.gandhi@excellenceinfonet.com> wrote:
>
> > Hi,
> >
> > Does kafka record the timestamp of the incoming message in its data log?
> I
> > checked one of the partition log and I can see the message without any
> > timestamp. Also there are few special characters in the message log. IS
> > that normal?
> >
> > Here is a sample log: pastebin.com/hStyCW13
> <http://pastebin.com/hStyCW13>
> >
> > Thanks,
> > Parth Gandhi
> > DevOps
> >
> > Disclaimer
> >
> > The information contained in this communication from the sender is
> > confidential. It is intended solely for use by the recipient and others
> > authorized to receive it. If you are not the recipient, you are hereby
> > notified that any disclosure, copying, distribution or taking action in
> > relation of the contents of this information is strictly prohibited and
> may
> > be unlawful.
> >
> > This email has been scanned for viruses and malware, and may have been
> > automatically archived by Mimecast Ltd, an innovator in Software as a
> > Service (SaaS) for business. Providing a safer and more useful place for
> > your human generated data. Specializing in; Security, archiving and
> > compliance. To find out more visit the Mimecast website.
> >
>
Disclaimer
The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.
This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd, an innovator in Software as a Service (SaaS) for business. Providing a safer and more useful place for your human generated data. Specializing in; Security, archiving and compliance. To find out more visit the Mimecast website.
Re: Kafka data log timestamp
Posted by Robin Moffatt <ro...@confluent.io>.
You can use kafkacat to examine the timestamp (and other metadata). Here's
an example of calling it, and two sample output records:
$ kafkacat -b localhost:9092 -t mysql_users -C -c2 -f '\nKey (%K bytes):
%k\t\nValue (%S bytes): %s\nTimestamp: %T\tPartition: %p\tOffset: %o\n--\n'
Key (1 bytes): 1
Value (79 bytes):
{"uid":1,"name":"Cliff","locale":"en_US","address_city":"St
Louis","elite":"P"}
Timestamp: 1520618381093 Partition: 0 Offset: 0
--
Key (1 bytes): 2
Value (79 bytes):
{"uid":2,"name":"Nick","locale":"en_US","address_city":"Palo
Alto","elite":"G"}
Timestamp: 1520618381093 Partition: 0 Offset: 1
--
--
Robin Moffatt | Developer Advocate | robin@confluent.io | @rmoff
On Thu, 10 Jan 2019 at 10:13, Parth Gandhi <
parth.gandhi@excellenceinfonet.com> wrote:
> Hi,
>
> Does kafka record the timestamp of the incoming message in its data log? I
> checked one of the partition log and I can see the message without any
> timestamp. Also there are few special characters in the message log. IS
> that normal?
>
> Here is a sample log: pastebin.com/hStyCW13
>
> Thanks,
> Parth Gandhi
> DevOps
>
> Disclaimer
>
> The information contained in this communication from the sender is
> confidential. It is intended solely for use by the recipient and others
> authorized to receive it. If you are not the recipient, you are hereby
> notified that any disclosure, copying, distribution or taking action in
> relation of the contents of this information is strictly prohibited and may
> be unlawful.
>
> This email has been scanned for viruses and malware, and may have been
> automatically archived by Mimecast Ltd, an innovator in Software as a
> Service (SaaS) for business. Providing a safer and more useful place for
> your human generated data. Specializing in; Security, archiving and
> compliance. To find out more visit the Mimecast website.
>