newbie: trouble configuring topic default retention (i.e. time to live)

[Andy Davidson <An...@SantaCruzIntegration.com>]

Hi 

I am new to kafka and need to pick up admin responsibilities for our
brokers. 

I am using kafka_2.11-0.10.0.0. I want to set up my broker so that topics
that are created automatically have a SLA of 1 hr. I.E. I want the data to
removed after 1 hr.

I added the following to my server.properties files and restarted the broker
retention.ms=3600000



I wrote some test code that published to a topic Œaedwip3¹. I did not create
the topic in advance. The aedwip3 topic does not have the expected default
values





$ bin/kafka-topics.sh --describe --zookeeper foo.com:2181 --topic aedwip3

Topic:aedwip3 PartitionCount:1 ReplicationFactor:1 Configs:

Topic: aedwip3 Partition: 0 Leader: 0 Replicas: 0 Isr: 0

$



If I use the command line I can explicitly set the TTL value (this topic was
altered when we where still using kafka_2.10-0.8.2.1



kafka-topics.sh --zookeeper foo.com:2181 -alter --topic footest --config
retention.ms=3600000



$ bin/kafka-topics.sh --describe --zookeeper foo.com:2181 --topic footest

Topic:footest PartitionCount:1 ReplicationFactor:1
Configs:retention.ms=3600000

Topic: footest Partition: 0 Leader: 0 Replicas: 0 Isr: 0

$ 



Any idea what my mistake is?



Thanks in advance



Andy

Re: newbie: trouble configuring topic default retention (i.e. time to live)

[Spico Florin <sp...@gmail.com>]

Hi!
  Regarding the configurations:
you are using the retention.ms property that should be used for a topic not
for the broker via server.properties. If you'd like to use at the broker
level then you have to use log.retention.ms.
Kafka separates the configuration for brokers and for topics. Please have a
look at the http://kafka.apache.org/documentation.html#configuration.
When you set up via the script you have used the proper property for topic (
retention.ms)
I hope that it helps.
 Regards,
 Florin




On Tue, Jul 5, 2016 at 8:26 PM, Andy Davidson <Andy@santacruzintegration.com
> wrote:

> Hi
>
> I am new to kafka and need to pick up admin responsibilities for our
> brokers.
>
> I am using kafka_2.11-0.10.0.0. I want to set up my broker so that topics
> that are created automatically have a SLA of 1 hr. I.E. I want the data to
> removed after 1 hr.
>
> I added the following to my server.properties files and restarted the
> broker
> retention.ms=3600000
>
>
>
> I wrote some test code that published to a topic Œaedwip3¹. I did not
> create
> the topic in advance. The aedwip3 topic does not have the expected default
> values
>
>
>
>
>
> $ bin/kafka-topics.sh --describe --zookeeper foo.com:2181 --topic aedwip3
>
> Topic:aedwip3 PartitionCount:1 ReplicationFactor:1 Configs:
>
> Topic: aedwip3 Partition: 0 Leader: 0 Replicas: 0 Isr: 0
>
> $
>
>
>
> If I use the command line I can explicitly set the TTL value (this topic
> was
> altered when we where still using kafka_2.10-0.8.2.1
>
>
>
> kafka-topics.sh --zookeeper foo.com:2181 -alter --topic footest --config
> retention.ms=3600000
>
>
>
> $ bin/kafka-topics.sh --describe --zookeeper foo.com:2181 --topic footest
>
> Topic:footest PartitionCount:1 ReplicationFactor:1
> Configs:retention.ms=3600000
>
> Topic: footest Partition: 0 Leader: 0 Replicas: 0 Isr: 0
>
> $
>
>
>
> Any idea what my mistake is?
>
>
>
> Thanks in advance
>
>
>
> Andy
>
>
>