You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Meghana Narasimhan <mn...@bandwidth.com> on 2017/01/20 16:20:11 UTC
timeindex file timestamp mismatch (0.10.1.0)
Hi,
I'm testing upgrading our cluster from 0.9.0.1 to 0.10.1.0 on 2 clusters A
and B. I have upgraded only the inter.broker.protocol.version to 0.10.1.0.
The log.message.format.version is still 0.9.0.1.
I'm writing test data from a java producer to the upgraded cluster A. As
expected the .timeindex files get created. I understand that the timestamp
will be appended to offsets only when I upgrade the
log.message.format.version. But when I run the following command, I get the
following.
kafka-run-class kafka.tools.DumpLogSegments --files
/mnt/data/kafka-logs/upgrade_test-2/00000000000014500426.timeindex
Dumping /mnt/data/kafka-logs/upgrade_test-2/00000000000014500426.timeindex
timestamp: 0 offset: 14500426
Found timestamp mismatch in
:/mnt/data/kafka-logs/upgrade_test-2/00000000000014500426.timeindex
Index timestamp: 0, log timestamp: -1
Index timestamp: 0, log timestamp: -1
Found out of order timestamp in
:/mnt/data/kafka-logs/upgrade_test-2/00000000000014500426.timeindex
Index timestamp: 0, Previously indexed timestamp: 0
What does the timestamp mismatch indicate ? is it an error ?
Also the other cluster, Cluster B is mirroring data from cluster A, and
when the run the dumplogSegment command on that cluster I get something
like this..
timestamp: 0 offset: 1124701
timestamp: 0 offset: 1124701
timestamp: 0 offset: 1124701
timestamp: 0 offset: 1124701
timestamp: 0 offset: 1124701
timestamp: 0 offset: 1124701
timestamp: 0 offset: 1124701
timestamp: 0 offset: 1124701
Any inputs will be of great help.
Thanks,
Meghana
Re: timeindex file timestamp mismatch (0.10.1.0)
Posted by Meghana Narasimhan <mn...@bandwidth.com>.
Thanks Sudev, but thats just the Jira for implementing the KIP. not sure it
addresses the issue that i am seeing.
Just trying to understand what the output means.
Thanks,
Meghana
On Fri, Jan 20, 2017 at 4:17 PM, Sudev A C <su...@goibibo.com> wrote:
> Hi,
>
> I think you are facing this issue.
> https://issues.apache.org/jira/browse/KAFKA-3163
>
> Thanks
> Sudev
>
> On Fri, Jan 20, 2017 at 9:50 PM Meghana Narasimhan <
> mnarasimhan@bandwidth.com> wrote:
>
> > Hi,
> > I'm testing upgrading our cluster from 0.9.0.1 to 0.10.1.0 on 2 clusters
> A
> > and B. I have upgraded only the inter.broker.protocol.version to
> 0.10.1.0.
> > The log.message.format.version is still 0.9.0.1.
> >
> > I'm writing test data from a java producer to the upgraded cluster A. As
> > expected the .timeindex files get created. I understand that the
> timestamp
> > will be appended to offsets only when I upgrade the
> > log.message.format.version. But when I run the following command, I get
> the
> > following.
> >
> > kafka-run-class kafka.tools.DumpLogSegments --files
> > /mnt/data/kafka-logs/upgrade_test-2/00000000000014500426.timeindex
> > Dumping /mnt/data/kafka-logs/upgrade_test-2/00000000000014500426.
> timeindex
> > timestamp: 0 offset: 14500426
> > Found timestamp mismatch in
> > :/mnt/data/kafka-logs/upgrade_test-2/00000000000014500426.timeindex
> > Index timestamp: 0, log timestamp: -1
> > Index timestamp: 0, log timestamp: -1
> > Found out of order timestamp in
> > :/mnt/data/kafka-logs/upgrade_test-2/00000000000014500426.timeindex
> > Index timestamp: 0, Previously indexed timestamp: 0
> >
> > What does the timestamp mismatch indicate ? is it an error ?
> >
> > Also the other cluster, Cluster B is mirroring data from cluster A, and
> > when the run the dumplogSegment command on that cluster I get something
> > like this..
> >
> >
> > timestamp: 0 offset: 1124701
> > timestamp: 0 offset: 1124701
> > timestamp: 0 offset: 1124701
> > timestamp: 0 offset: 1124701
> > timestamp: 0 offset: 1124701
> > timestamp: 0 offset: 1124701
> > timestamp: 0 offset: 1124701
> > timestamp: 0 offset: 1124701
> >
> > Any inputs will be of great help.
> > Thanks,
> > Meghana
> >
>
Re: timeindex file timestamp mismatch (0.10.1.0)
Posted by Sudev A C <su...@goibibo.com>.
Hi,
I think you are facing this issue.
https://issues.apache.org/jira/browse/KAFKA-3163
Thanks
Sudev
On Fri, Jan 20, 2017 at 9:50 PM Meghana Narasimhan <
mnarasimhan@bandwidth.com> wrote:
> Hi,
> I'm testing upgrading our cluster from 0.9.0.1 to 0.10.1.0 on 2 clusters A
> and B. I have upgraded only the inter.broker.protocol.version to 0.10.1.0.
> The log.message.format.version is still 0.9.0.1.
>
> I'm writing test data from a java producer to the upgraded cluster A. As
> expected the .timeindex files get created. I understand that the timestamp
> will be appended to offsets only when I upgrade the
> log.message.format.version. But when I run the following command, I get the
> following.
>
> kafka-run-class kafka.tools.DumpLogSegments --files
> /mnt/data/kafka-logs/upgrade_test-2/00000000000014500426.timeindex
> Dumping /mnt/data/kafka-logs/upgrade_test-2/00000000000014500426.timeindex
> timestamp: 0 offset: 14500426
> Found timestamp mismatch in
> :/mnt/data/kafka-logs/upgrade_test-2/00000000000014500426.timeindex
> Index timestamp: 0, log timestamp: -1
> Index timestamp: 0, log timestamp: -1
> Found out of order timestamp in
> :/mnt/data/kafka-logs/upgrade_test-2/00000000000014500426.timeindex
> Index timestamp: 0, Previously indexed timestamp: 0
>
> What does the timestamp mismatch indicate ? is it an error ?
>
> Also the other cluster, Cluster B is mirroring data from cluster A, and
> when the run the dumplogSegment command on that cluster I get something
> like this..
>
>
> timestamp: 0 offset: 1124701
> timestamp: 0 offset: 1124701
> timestamp: 0 offset: 1124701
> timestamp: 0 offset: 1124701
> timestamp: 0 offset: 1124701
> timestamp: 0 offset: 1124701
> timestamp: 0 offset: 1124701
> timestamp: 0 offset: 1124701
>
> Any inputs will be of great help.
> Thanks,
> Meghana
>
Re: timeindex file timestamp mismatch (0.10.1.0)
Posted by Meghana Narasimhan <mn...@bandwidth.com>.
Thanks for the reply and explanation, Jun ! So yes, I was running the
DumpLogSegment tool on an active timeindex segment. But among two clusters
i tested with, I did not get the same error on the destination clutser
(which is mirroring data from source cluster) even when I ran the
DumpLogSegment tool on an active timeindex file. So thats what got me
curious.
10485756 Jan 23 15:56 00000000000001690648.timeindex [this is the active
timeindex]
$ sudo kafka-run-class kafka.tools.DumpLogSegments --files
/mnt/data/kafka-logs/upgrade_test-0/00000000000001690648.timeindex
Dumping /mnt/data/kafka-logs/upgrade_test-0/00000000000001690648.timeindex
timestamp: 0 offset: 1690648
timestamp: 0 offset: 1690648
timestamp: 0 offset: 1690648
timestamp: 0 offset: 1690648
timestamp: 0 offset: 1690648
Thanks,
Meghana
On Fri, Jan 20, 2017 at 5:07 PM, Jun Rao <ju...@confluent.io> wrote:
> Meghana,
>
> You are probably seeing this when running the DumpLogSegment tool on the
> active (last) log segment. DumpLogSegment tool is supposed to only be used
> on the index of the immutable segments (i.e., when they are rolled). We
> preallocate the index on the active segment with 0 values. So,
> DumpLogSegment doesn't know the actual valid entries in the index and will
> report those 0 values as errors. If you wait for the log segment to roll,
> you should see the timeindex resized to 0 length.
>
> Thanks,
>
> Jun
>
> On Fri, Jan 20, 2017 at 8:20 AM, Meghana Narasimhan <
> mnarasimhan@bandwidth.com> wrote:
>
> > Hi,
> > I'm testing upgrading our cluster from 0.9.0.1 to 0.10.1.0 on 2 clusters
> A
> > and B. I have upgraded only the inter.broker.protocol.version to
> 0.10.1.0.
> > The log.message.format.version is still 0.9.0.1.
> >
> > I'm writing test data from a java producer to the upgraded cluster A. As
> > expected the .timeindex files get created. I understand that the
> timestamp
> > will be appended to offsets only when I upgrade the
> > log.message.format.version. But when I run the following command, I get
> the
> > following.
> >
> > kafka-run-class kafka.tools.DumpLogSegments --files
> > /mnt/data/kafka-logs/upgrade_test-2/00000000000014500426.timeindex
> > Dumping /mnt/data/kafka-logs/upgrade_test-2/00000000000014500426.
> timeindex
> > timestamp: 0 offset: 14500426
> > Found timestamp mismatch in
> > :/mnt/data/kafka-logs/upgrade_test-2/00000000000014500426.timeindex
> > Index timestamp: 0, log timestamp: -1
> > Index timestamp: 0, log timestamp: -1
> > Found out of order timestamp in
> > :/mnt/data/kafka-logs/upgrade_test-2/00000000000014500426.timeindex
> > Index timestamp: 0, Previously indexed timestamp: 0
> >
> > What does the timestamp mismatch indicate ? is it an error ?
> >
> > Also the other cluster, Cluster B is mirroring data from cluster A, and
> > when the run the dumplogSegment command on that cluster I get something
> > like this..
> >
> >
> > timestamp: 0 offset: 1124701
> > timestamp: 0 offset: 1124701
> > timestamp: 0 offset: 1124701
> > timestamp: 0 offset: 1124701
> > timestamp: 0 offset: 1124701
> > timestamp: 0 offset: 1124701
> > timestamp: 0 offset: 1124701
> > timestamp: 0 offset: 1124701
> >
> > Any inputs will be of great help.
> > Thanks,
> > Meghana
> >
>
Re: timeindex file timestamp mismatch (0.10.1.0)
Posted by Jun Rao <ju...@confluent.io>.
Meghana,
You are probably seeing this when running the DumpLogSegment tool on the
active (last) log segment. DumpLogSegment tool is supposed to only be used
on the index of the immutable segments (i.e., when they are rolled). We
preallocate the index on the active segment with 0 values. So,
DumpLogSegment doesn't know the actual valid entries in the index and will
report those 0 values as errors. If you wait for the log segment to roll,
you should see the timeindex resized to 0 length.
Thanks,
Jun
On Fri, Jan 20, 2017 at 8:20 AM, Meghana Narasimhan <
mnarasimhan@bandwidth.com> wrote:
> Hi,
> I'm testing upgrading our cluster from 0.9.0.1 to 0.10.1.0 on 2 clusters A
> and B. I have upgraded only the inter.broker.protocol.version to 0.10.1.0.
> The log.message.format.version is still 0.9.0.1.
>
> I'm writing test data from a java producer to the upgraded cluster A. As
> expected the .timeindex files get created. I understand that the timestamp
> will be appended to offsets only when I upgrade the
> log.message.format.version. But when I run the following command, I get the
> following.
>
> kafka-run-class kafka.tools.DumpLogSegments --files
> /mnt/data/kafka-logs/upgrade_test-2/00000000000014500426.timeindex
> Dumping /mnt/data/kafka-logs/upgrade_test-2/00000000000014500426.timeindex
> timestamp: 0 offset: 14500426
> Found timestamp mismatch in
> :/mnt/data/kafka-logs/upgrade_test-2/00000000000014500426.timeindex
> Index timestamp: 0, log timestamp: -1
> Index timestamp: 0, log timestamp: -1
> Found out of order timestamp in
> :/mnt/data/kafka-logs/upgrade_test-2/00000000000014500426.timeindex
> Index timestamp: 0, Previously indexed timestamp: 0
>
> What does the timestamp mismatch indicate ? is it an error ?
>
> Also the other cluster, Cluster B is mirroring data from cluster A, and
> when the run the dumplogSegment command on that cluster I get something
> like this..
>
>
> timestamp: 0 offset: 1124701
> timestamp: 0 offset: 1124701
> timestamp: 0 offset: 1124701
> timestamp: 0 offset: 1124701
> timestamp: 0 offset: 1124701
> timestamp: 0 offset: 1124701
> timestamp: 0 offset: 1124701
> timestamp: 0 offset: 1124701
>
> Any inputs will be of great help.
> Thanks,
> Meghana
>