Re: [EXTERNAL] Re: Permanent topic

["Singh, Harpreet" <ha...@hbc.com>]

Hi,
According to cloudera manager kafka documentation:
log.retention.ms: The maximum time before a new log segment is rolled out.
If both log.retention.ms and log.retention.bytes are set, a segment is
deleted when either limit is exceeded. The special value of -1 is
interpreted as unlimited. This property is used in Kafka 1.4.0 and later in
place of log.retention.hours.


Regards,
Harpreet Singh

On Fri, Mar 15, 2019 at 1:58 PM M. Manna <ma...@gmail.com> wrote:

> There is difference between retention.bytes and retention.ms. Yes,
> retention.bytes can be set to -1, but nowhere in the docs says about
> retention.ms = -1
>
> It might be possible that using -1 it's accepting as a "Value" - which
> means it might not be validating. But it's not on official docs. I would
> not be using this if official docs isn't supporting the same.
>
> Could anyone else shed some light on this?
>
> Thanks,
>
>
>
> On Fri, 15 Mar 2019 at 17:27, Maxim Kheyfets <ma...@clearme.com>
> wrote:
>
> > Good time of the day,
> >
> > We are using kafka 1.0.1, and want to create a permanent topic. One
> online
> > post suggests setting retention.ms and retention.bytes to -1. The sample
> > below shows system accepts  -1 correctly, but I don't see this documented
> > anywhere explicitly in the official documentation.
> >
> > Could you confirm, and/or point me to the right official page?
> >
> > Thank you,
> > Maxim
> >
> >
> > kafka-topics.sh --create --zookeeper zk.local/kafka --replication-factor
> 3
> > --partitions 30 --topic maxim-test
> > kafka-configs.sh --zookeeper zk.local/kafka --entity-type topics
> > --entity-name maxim-test --alter --add-config retention.ms=-1
> > kafka-configs.sh --zookeeper zk.local/kafka --entity-type topics
> > --entity-name maxim-test --alter --add-config retention.bytes=-1
> >
> > --describe shows it as successful:
> > zk.local
> >
> > Topic:maxim-test PartitionCount:30 ReplicationFactor:3
> > Configs:retention.ms=-1,retention.bytes=-1
> > Topic: msg-opt-in Partition: 0 Leader: 1 Replicas: 1,2,3 Isr: 1,2,3
> Topic:
> > msg-opt-in Partition: 1 Leader: 2 Replicas: 2,3,1 Isr: 2,3,1 Topic:
> > msg-opt-in Partition: 2 Leader: 3 Replicas: 3,1,2 Isr: 3,1,2 Topic:
> > msg-opt-in Partition: 3 Leader: 1 Replicas: 1,3,2 Isr: 1,3,2 Topic:
> > msg-opt-in Partition: 4 Leader: 2 Replicas: 2,1,3 Isr: 2,1,3 Topic:
> > msg-opt-in Partition: 5 Leader: 3 Replicas: 3,2,1 Isr: 3,2,1 Topic:
> > msg-opt-in Partition: 6 Leader: 1 Replicas: 1,2,3 Isr: 1,2,3 Topic:
> > msg-opt-in Partition: 7 Leader: 2 Replicas: 2,3,1 Isr: 2,3,1 Topic:
> > msg-opt-in Partition: 8 Leader: 3 Replicas: 3,1,2 Isr: 3,1,2 Topic:
> > msg-opt-in Partition: 9 Leader: 1 Replicas: 1,3,2 Isr: 1,3,2 Topic:
> > msg-opt-in Partition: 10 Leader: 2 Replicas: 2,1,3 Isr: 2,1,3 Topic:
> > msg-opt-in Partition: 11 Leader: 3 Replicas: 3,2,1 Isr: 3,2,1 Topic:
> > msg-opt-in Partition: 12 Leader: 1 Replicas: 1,2,3 Isr: 1,2,3 Topic:
> > msg-opt-in Partition: 13 Leader: 2 Replicas: 2,3,1 Isr: 2,3,1 Topic:
> > msg-opt-in Partition: 14 Leader: 3 Replicas: 3,1,2 Isr: 3,1,2 Topic:
> > msg-opt-in Partition: 15 Leader: 1 Replicas: 1,3,2 Isr: 1,3,2 Topic:
> > msg-opt-in Partition: 16 Leader: 2 Replicas: 2,1,3 Isr: 2,1,3 Topic:
> > msg-opt-in Partition: 17 Leader: 3 Replicas: 3,2,1 Isr: 3,2,1 Topic:
> > msg-opt-in Partition: 18 Leader: 1 Replicas: 1,2,3 Isr: 1,2,3 Topic:
> > msg-opt-in Partition: 19 Leader: 2 Replicas: 2,3,1 Isr: 2,3,1 Topic:
> > msg-opt-in Partition: 20 Leader: 3 Replicas: 3,1,2 Isr: 3,1,2 Topic:
> > msg-opt-in Partition: 21 Leader: 1 Replicas: 1,3,2 Isr: 1,3,2 Topic:
> > msg-opt-in Partition: 22 Leader: 2 Replicas: 2,1,3 Isr: 2,1,3 Topic:
> > msg-opt-in Partition: 23 Leader: 3 Replicas: 3,2,1 Isr: 3,2,1 Topic:
> > msg-opt-in Partition: 24 Leader: 1 Replicas: 1,2,3 Isr: 1,2,3 Topic:
> > msg-opt-in Partition: 25 Leader: 2 Replicas: 2,3,1 Isr: 2,3,1 Topic:
> > msg-opt-in Partition: 26 Leader: 3 Replicas: 3,1,2 Isr: 3,1,2 Topic:
> > msg-opt-in Partition: 27 Leader: 1 Replicas: 1,3,2 Isr: 1,3,2 Topic:
> > msg-opt-in Partition: 28 Leader: 2 Replicas: 2,1,3 Isr: 2,1,3 Topic:
> > msg-opt-in Partition: 29 Leader: 3 Replicas: 3,2,1 Isr: 3,2,1
> > --
> >
> >
> >
> >
> > *Maxim Kheyfets*
> > Senior DevOps Engineer
> >
> > maxim.kheyfets@clearme.com |
> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.clearme.com&d=DwIBaQ&c=T4LuzJg_R6QwRnqJoo4xTCUXoKbdWTdhZj7r4OYEklY&r=LvsTbNOOBqkmhXCFn1QQ5IEQuGkvpWEqGs0cuadvTrw&m=GVHaG2_fBmuKp3BtO6Jo9hbRbNtKd2HBoZt-e612NAw&s=c8TyGxG5Biv0hzLPlDLpf8SEP0MSnxgUd_VhGQ4o_4Y&e=
> >
>