You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@kafka.apache.org by Parth Gandhi <pa...@excellenceinfonet.com> on 2019/06/18 10:28:39 UTC

Kafka broker server high memory utilization

Hi,

I am seeing my kafka broker server running at >90% memory and most of the
memory is used by buffer/cache. Is this a normal expected behavior? or
shall i look at increasing the Physical memory of the servers.
[image: image.png]
PS: I have 3 broker server with around 30+ topic and 400+ partition spread
across these servers.

Thanks,
Parth Gandhi
DevOps

Disclaimer

The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.

This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd, an innovator in Software as a Service (SaaS) for business. Providing a safer and more useful place for your human generated data. Specializing in; Security, archiving and compliance. To find out more visit the Mimecast website.

Re: Kafka broker server high memory utilization

Posted by Parth Gandhi <pa...@excellenceinfonet.com>.

Thanks Steve. So i shouldn't worry about increasing the RAM at this moment.
Also do you think that the system resource are OK to handle the workload.

3 server  - 2 core processor and 8 GB Ram. Retention policy is set to 168
hours.

Thanks,
Parth Gandhi
DevOps

On Tue, Jun 18, 2019 at 12:07 PM Steve Howard <st...@confluent.io>
wrote:

> Hi Parth,
>
> Yes, this is by design. See the following URL, paying particularly close
> attention to the section on memory.
>
> https://docs.confluent.io/current/kafka/deployment.html
> <https://docs.confluent.io/current/kafka/deployment.html>
>
> Thanks,
>
> Steve
>
> On Tue, Jun 18, 2019 at 6:29 AM Parth Gandhi <
> parth.gandhi@excellenceinfonet.com> wrote:
>
> > Hi,
> >
> > I am seeing my kafka broker server running at >90% memory and most of the
> > memory is used by buffer/cache. Is this a normal expected behavior? or
> > shall i look at increasing the Physical memory of the servers.
> > [image: image.png]
> > PS: I have 3 broker server with around 30+ topic and 400+ partition
> spread
> > across these servers.
> >
> > Thanks,
> > Parth Gandhi
> > DevOps
> >
> >
> > *Disclaimer*
> >
> > The information contained in this communication from the sender is
> > confidential. It is intended solely for use by the recipient and others
> > authorized to receive it. If you are not the recipient, you are hereby
> > notified that any disclosure, copying, distribution or taking action in
> > relation of the contents of this information is strictly prohibited and
> may
> > be unlawful.
> >
> > This email has been scanned for viruses and malware, and may have been
> > automatically archived by *Mimecast Ltd*, an innovator in Software as a
> > Service (SaaS) for business. Providing a *safer* and *more useful* place
> > for your human generated data. Specializing in; Security, archiving and
> > compliance. To find out more Click Here
> > <http://www.mimecast.com/products/
> <http://www.mimecast.com/products/>
> >.
> >
>

Disclaimer

The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.

This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd, an innovator in Software as a Service (SaaS) for business. Providing a safer and more useful place for your human generated data. Specializing in; Security, archiving and compliance. To find out more visit the Mimecast website.

Re: Kafka broker server high memory utilization

Posted by Steve Howard <st...@confluent.io>.

Hi Parth,

Yes, this is by design.  See the following URL, paying particularly close
attention to the section on memory.

https://docs.confluent.io/current/kafka/deployment.html

Thanks,

Steve

On Tue, Jun 18, 2019 at 6:29 AM Parth Gandhi <
parth.gandhi@excellenceinfonet.com> wrote:

> Hi,
>
> I am seeing my kafka broker server running at >90% memory and most of the
> memory is used by buffer/cache. Is this a normal expected behavior? or
> shall i look at increasing the Physical memory of the servers.
> [image: image.png]
> PS: I have 3 broker server with around 30+ topic and 400+ partition spread
> across these servers.
>
> Thanks,
> Parth Gandhi
> DevOps
>
>
> *Disclaimer*
>
> The information contained in this communication from the sender is
> confidential. It is intended solely for use by the recipient and others
> authorized to receive it. If you are not the recipient, you are hereby
> notified that any disclosure, copying, distribution or taking action in
> relation of the contents of this information is strictly prohibited and may
> be unlawful.
>
> This email has been scanned for viruses and malware, and may have been
> automatically archived by *Mimecast Ltd*, an innovator in Software as a
> Service (SaaS) for business. Providing a *safer* and *more useful* place
> for your human generated data. Specializing in; Security, archiving and
> compliance. To find out more Click Here
> <http://www.mimecast.com/products/>.
>