You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Daniele Ascione (JIRA)" <ji...@apache.org> on 2019/02/12 15:25:00 UTC
[jira] [Comment Edited] (KAFKA-7794) kafka.tools.GetOffsetShell
does not return the offset in some cases
[ https://issues.apache.org/jira/browse/KAFKA-7794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16766139#comment-16766139 ]
Daniele Ascione edited comment on KAFKA-7794 at 2/12/19 3:24 PM:
-----------------------------------------------------------------
[~kartikvk1996], I am able to reproduce the behaviour on my machine. I am using Kafka in Red Hat Enterprise Linux Server release 7.3 (Maipo).
You could produce more messages using:
{code:java}
bin/kafka-producer-perf-test.sh --topic demo --throughput 10 --num-records 50000 --record-size 5 --producer-props bootstrap.servers=127.0.0.1:9092
{code}
I executed it in my system right now, then I typed:
{code:java}
bin/kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list localhost:9092 --topic test --time 1549983817353
{code}
and the output is:
!image-2019-02-12-16-19-25-170.png!
I have some messages after and before that timestamp:
!image-2019-02-12-16-23-38-399.png!
was (Author: dascione):
[~kartikvk1996], I am able to reproduce the behaviour on my machine. I am using Kafka in Red Hat Enterprise Linux Server release 7.3 (Maipo).
You could produce more messages using:
{code:java}
bin/kafka-producer-perf-test.sh --topic demo --throughput 10 --num-records 50000 --record-size 5 --producer-props bootstrap.servers=127.0.0.1:9092
{code}
If you then take a message using the procedure I described, you should have.
I executed in my system right now, then I typed:
{code:java}
bin/kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list localhost:9092 --topic test --time 1549983817353
{code}
and the output is:
!image-2019-02-12-16-19-25-170.png!
I have some messages after and before that timestamp:
!image-2019-02-12-16-23-38-399.png!
> kafka.tools.GetOffsetShell does not return the offset in some cases
> -------------------------------------------------------------------
>
> Key: KAFKA-7794
> URL: https://issues.apache.org/jira/browse/KAFKA-7794
> Project: Kafka
> Issue Type: Bug
> Components: tools
> Affects Versions: 0.10.2.0, 0.10.2.1, 0.10.2.2
> Reporter: Daniele Ascione
> Assignee: Kartik
> Priority: Critical
> Labels: Kafka, ShellCommands, kafka-0.10, offset, shell, shell-script, shellscript, tools, usability
> Attachments: image-2019-02-11-20-51-07-805.png, image-2019-02-11-20-56-13-362.png, image-2019-02-11-20-57-03-579.png, image-2019-02-12-16-19-25-170.png, image-2019-02-12-16-21-13-126.png, image-2019-02-12-16-23-38-399.png
>
>
> For some input for the timestamps (different from -1 or -2) the GetOffset is not able to retrieve the offset.
> For example, if _x_ is the timestamp in that "not working range", and you execute:
> {code:java}
> bin/kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list $KAFKA_ADDRESS --topic $MY_TOPIC --time x
> {code}
> The output is:
> {code:java}
> MY_TOPIC:8:
> MY_TOPIC:2:
> MY_TOPIC:5:
> MY_TOPIC:4:
> MY_TOPIC:7:
> MY_TOPIC:1:
> MY_TOPIC:9:{code}
> while after the last ":" an integer representing the offset is expected.
> ----
> Steps to reproduce it:
> # Consume all the messages from the beginning and print the timestamp:
> {code:java}
> bin/kafka-simple-consumer-shell.sh --no-wait-at-logend --broker-list $KAFKA_ADDRESS --topic $MY_TOPIC --property print.timestamp=true > messages{code}
> # Sort the messages by timestamp and get some of the oldest messages:
> {code:java}
> awk -F "CreateTime:" '{ print $2}' messages | sort -n > msg_sorted{code}
> # Take (for example) the timestamp of the 10th oldest message, and see if GetOffsetShell is not able to print the offset:
> {code:java}
> timestamp="$(sed '10q;d' msg_sorted | cut -f1)"
> bin/kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list $KAFKA_ADDRESS --topic $MY_TOPIC --time $timestamp
> # The output should be something like:
> # MY_TOPIC:1:
> # MY_TOPIC:2:
> (repeated for every partition){code}
> # Verify that the message with that timestamp is still in Kafka:
> {code:java}
> bin/kafka-simple-consumer-shell.sh --no-wait-at-logend --broker-list $KAFKA_ADDRESS --topic $MY_TOPIC --property print.timestamp=true | grep "CreateTime:$timestamp" {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)