You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Monika Garg <ga...@gmail.com> on 2014/01/23 11:20:08 UTC
Kafka request.log size is increasing continuously
Hi
I have created one node kafka cluster and one topic name test1.
Then I started ConsoleProducer and ConsoleConsumer.Both started
successfully.But what I see is in the /kafka/kafka_2.8.0-0.8.0/logs/
directory
-rw-rw-r-- 1 admin admin 5203 Jan 23 15:37 controller.log
*-rw-rw-r-- 1 admin admin 2589466 Jan 23 15:45 kafka-request.log*
-rw-rw-r-- 1 admin admin 2604 Jan 23 15:44 kafkaServer-gc.log
-rw-rw-r-- 1 admin admin 6505 Jan 23 15:40 server.log
-rw-rw-r-- 1 admin admin 2167 Jan 23 15:37 state-change.log
The size of file *kafka-request.log* is continuously increasing even though
I am not giving any message to producer.
When I killed ConsoleConsumer and ConsoleProducer,the size of file
*kafka-request.log
*stopped increasing.
So I want to know the reason why this is(*kafka-request.log) *is only file
whose size is getting increased continuously even though no message is
given to produce or no message is consumed by consumer.
--
*Moniii*
Re: Kafka request.log size is increasing continuously
Posted by Guozhang Wang <wa...@gmail.com>.
The producer/consumer log compaction/retention can be set in log4j.
Guozhang
On Thu, Jan 23, 2014 at 7:57 AM, Monika Garg <ga...@gmail.com> wrote:
> It means my free disk size will keep on reducing till Producer and Consumer
> will be running?Is there any way to stop the increment in file size
> (kafka-request.log)?
>
>
> On Thu, Jan 23, 2014 at 8:37 PM, Jun Rao <ju...@gmail.com> wrote:
>
> > Well, the consumer keeps sending fetch requests even when there are no
> more
> > messages. Even fetch request will only block on the broker for up to
> > max.wait.time. Also, the producer periodically issues metadata requests,
> > which are also logged in kafka-request.log.
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Thu, Jan 23, 2014 at 2:20 AM, Monika Garg <ga...@gmail.com>
> wrote:
> >
> > > Hi
> > >
> > > I have created one node kafka cluster and one topic name test1.
> > > Then I started ConsoleProducer and ConsoleConsumer.Both started
> > > successfully.But what I see is in the /kafka/kafka_2.8.0-0.8.0/logs/
> > > directory
> > >
> > > -rw-rw-r-- 1 admin admin 5203 Jan 23 15:37 controller.log
> > > *-rw-rw-r-- 1 admin admin 2589466 Jan 23 15:45 kafka-request.log*
> > > -rw-rw-r-- 1 admin admin 2604 Jan 23 15:44 kafkaServer-gc.log
> > > -rw-rw-r-- 1 admin admin 6505 Jan 23 15:40 server.log
> > > -rw-rw-r-- 1 admin admin 2167 Jan 23 15:37 state-change.log
> > >
> > > The size of file *kafka-request.log* is continuously increasing even
> > though
> > > I am not giving any message to producer.
> > > When I killed ConsoleConsumer and ConsoleProducer,the size of file
> > > *kafka-request.log
> > > *stopped increasing.
> > > So I want to know the reason why this is(*kafka-request.log) *is only
> > file
> > > whose size is getting increased continuously even though no message is
> > > given to produce or no message is consumed by consumer.
> > >
> > >
> > > --
> > > *Moniii*
> > >
> >
>
>
>
> --
> *Moniii*
>
--
-- Guozhang
Re: Kafka request.log size is increasing continuously
Posted by Monika Garg <ga...@gmail.com>.
It means my free disk size will keep on reducing till Producer and Consumer
will be running?Is there any way to stop the increment in file size
(kafka-request.log)?
On Thu, Jan 23, 2014 at 8:37 PM, Jun Rao <ju...@gmail.com> wrote:
> Well, the consumer keeps sending fetch requests even when there are no more
> messages. Even fetch request will only block on the broker for up to
> max.wait.time. Also, the producer periodically issues metadata requests,
> which are also logged in kafka-request.log.
>
> Thanks,
>
> Jun
>
>
> On Thu, Jan 23, 2014 at 2:20 AM, Monika Garg <ga...@gmail.com> wrote:
>
> > Hi
> >
> > I have created one node kafka cluster and one topic name test1.
> > Then I started ConsoleProducer and ConsoleConsumer.Both started
> > successfully.But what I see is in the /kafka/kafka_2.8.0-0.8.0/logs/
> > directory
> >
> > -rw-rw-r-- 1 admin admin 5203 Jan 23 15:37 controller.log
> > *-rw-rw-r-- 1 admin admin 2589466 Jan 23 15:45 kafka-request.log*
> > -rw-rw-r-- 1 admin admin 2604 Jan 23 15:44 kafkaServer-gc.log
> > -rw-rw-r-- 1 admin admin 6505 Jan 23 15:40 server.log
> > -rw-rw-r-- 1 admin admin 2167 Jan 23 15:37 state-change.log
> >
> > The size of file *kafka-request.log* is continuously increasing even
> though
> > I am not giving any message to producer.
> > When I killed ConsoleConsumer and ConsoleProducer,the size of file
> > *kafka-request.log
> > *stopped increasing.
> > So I want to know the reason why this is(*kafka-request.log) *is only
> file
> > whose size is getting increased continuously even though no message is
> > given to produce or no message is consumed by consumer.
> >
> >
> > --
> > *Moniii*
> >
>
--
*Moniii*
Re: Kafka request.log size is increasing continuously
Posted by Jun Rao <ju...@gmail.com>.
Well, the consumer keeps sending fetch requests even when there are no more
messages. Even fetch request will only block on the broker for up to
max.wait.time. Also, the producer periodically issues metadata requests,
which are also logged in kafka-request.log.
Thanks,
Jun
On Thu, Jan 23, 2014 at 2:20 AM, Monika Garg <ga...@gmail.com> wrote:
> Hi
>
> I have created one node kafka cluster and one topic name test1.
> Then I started ConsoleProducer and ConsoleConsumer.Both started
> successfully.But what I see is in the /kafka/kafka_2.8.0-0.8.0/logs/
> directory
>
> -rw-rw-r-- 1 admin admin 5203 Jan 23 15:37 controller.log
> *-rw-rw-r-- 1 admin admin 2589466 Jan 23 15:45 kafka-request.log*
> -rw-rw-r-- 1 admin admin 2604 Jan 23 15:44 kafkaServer-gc.log
> -rw-rw-r-- 1 admin admin 6505 Jan 23 15:40 server.log
> -rw-rw-r-- 1 admin admin 2167 Jan 23 15:37 state-change.log
>
> The size of file *kafka-request.log* is continuously increasing even though
> I am not giving any message to producer.
> When I killed ConsoleConsumer and ConsoleProducer,the size of file
> *kafka-request.log
> *stopped increasing.
> So I want to know the reason why this is(*kafka-request.log) *is only file
> whose size is getting increased continuously even though no message is
> given to produce or no message is consumed by consumer.
>
>
> --
> *Moniii*
>