Message Timeout

[Klaus Schaefers <kl...@ligatus.com>]

Hi,

I am new to Kafka and I have a question how Kafka handles scenarios where
no consumer is available. Can I configure Kafka in such a way that the
messages will be dropped after x seconds? Otherwise I would be afraid that
the queues would overflow...

Cheers,

Klaus



-- 

-- 

Klaus Schaefers
Senior Optimization Manager

Ligatus GmbH
Hohenstaufenring 30-32
D-50674 Köln

Tel.:  +49 (0) 221 / 56939 -784
Fax:  +49 (0) 221 / 56 939 - 599
E-Mail: klaus.schaefers@ligatus.com
Web: www.ligatus.de

HRB Köln 56003
Geschäftsführung:
Dipl.-Kaufmann Lars Hasselbach, Dipl.-Kaufmann Klaus Ludemann,
Dipl.-Wirtschaftsingenieur Arne Wolter

Re: Message Timeout

[Neha Narkhede <ne...@gmail.com>]

You can control retention using "log.retention.hours",
"log.retention.minutes" or "log.retention.bytes".


On Fri, Jun 27, 2014 at 2:06 AM, cacsar@gmail.com <ca...@gmail.com> wrote:

> Message retention in Kafka is disconnected from message consumption.
> Messages are all persisted to disk and the queues do not need to fit in RAM
> unlike some other systems. There are configuration values that control
> maximum log size in terms of MB and the duration of retention which is
> typically in terms of days, weeks or months, though perhaps hours at very
> high volumes. While you certainly could configure Kafka so that it would
> run out of disk space this can be avoided by a combination of configuration
> changes and bigger or more cheap spinning disks, or distributing the data
> across more machines.
>
> I hope this helps, though others likely have the configuration values at
> their fingertips.
>
> Christian
> On Jun 27, 2014 1:09 AM, "Klaus Schaefers" <kl...@ligatus.com>
> wrote:
>
> > Hi,
> >
> > I am new to Kafka and I have a question how Kafka handles scenarios where
> > no consumer is available. Can I configure Kafka in such a way that the
> > messages will be dropped after x seconds? Otherwise I would be afraid
> that
> > the queues would overflow...
> >
> > Cheers,
> >
> > Klaus
> >
> >
> >
> > --
> >
> > --
> >
> > Klaus Schaefers
> > Senior Optimization Manager
> >
> > Ligatus GmbH
> > Hohenstaufenring 30-32
> > D-50674 Köln
> >
> > Tel.:  +49 (0) 221 / 56939 -784
> > Fax:  +49 (0) 221 / 56 939 - 599
> > E-Mail: klaus.schaefers@ligatus.com
> > Web: www.ligatus.de
> >
> > HRB Köln 56003
> > Geschäftsführung:
> > Dipl.-Kaufmann Lars Hasselbach, Dipl.-Kaufmann Klaus Ludemann,
> > Dipl.-Wirtschaftsingenieur Arne Wolter
> >
>

Re: Message Timeout

["cacsar@gmail.com" <ca...@gmail.com>]

Message retention in Kafka is disconnected from message consumption.
Messages are all persisted to disk and the queues do not need to fit in RAM
unlike some other systems. There are configuration values that control
maximum log size in terms of MB and the duration of retention which is
typically in terms of days, weeks or months, though perhaps hours at very
high volumes. While you certainly could configure Kafka so that it would
run out of disk space this can be avoided by a combination of configuration
changes and bigger or more cheap spinning disks, or distributing the data
across more machines.

I hope this helps, though others likely have the configuration values at
their fingertips.

Christian
On Jun 27, 2014 1:09 AM, "Klaus Schaefers" <kl...@ligatus.com>
wrote:

> Hi,
>
> I am new to Kafka and I have a question how Kafka handles scenarios where
> no consumer is available. Can I configure Kafka in such a way that the
> messages will be dropped after x seconds? Otherwise I would be afraid that
> the queues would overflow...
>
> Cheers,
>
> Klaus
>
>
>
> --
>
> --
>
> Klaus Schaefers
> Senior Optimization Manager
>
> Ligatus GmbH
> Hohenstaufenring 30-32
> D-50674 Köln
>
> Tel.:  +49 (0) 221 / 56939 -784
> Fax:  +49 (0) 221 / 56 939 - 599
> E-Mail: klaus.schaefers@ligatus.com
> Web: www.ligatus.de
>
> HRB Köln 56003
> Geschäftsführung:
> Dipl.-Kaufmann Lars Hasselbach, Dipl.-Kaufmann Klaus Ludemann,
> Dipl.-Wirtschaftsingenieur Arne Wolter
>