You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Rahul Reddy <ra...@gmail.com> on 2019/02/06 14:19:21 UTC
Maximum memory usage
Hello,
I see maximum memory usage alerts in my system.log couple of times in a day
as INFO. So far I haven't seen any issue with db. Why those messages are
logged in system.log do we have any impact for reads/writes with those
warnings? And what nerd to be looked
INFO [RMI TCP Connection(170917)-127.0.0.1] 2019-02-05 23:15:47,408
NoSpamLogger.java:91 - Maximum memory usage reached (512.000MiB), cannot
allocate chunk of 1.000MiB
Thanks in advance
Re: Maximum memory usage
Posted by Rahul Reddy <ra...@gmail.com>.
One of the other db with 100mb partition* out of memory happens very
frequently.
```Percentile SSTables Write Latency Read Latency Partition
Size Cell Count
(micros) (micros) (bytes)
50% 0.00 0.00 0.00 182785
2759
75% 0.00 0.00 0.00 943127
14237
95% 0.00 0.00 0.00 4866323
73457
98% 0.00 0.00 0.00 12108970
152321
99% 0.00 0.00 0.00 17436917
219342
Min 0.00 0.00 0.00 150
2
Max 0.00 0.00 0.00 107964792
1358102
````
On Thu, Feb 7, 2019, 2:29 AM dinesh.joshi@yahoo.com.INVALID
<di...@yahoo.com.invalid> wrote:
> Are you running any nodetool commands during that period? IIRC, this is a
> log entry emitted by the BufferPool. It may be harm unless it's happening
> very often or logging a OOM.
>
> Dinesh
>
>
> On Wednesday, February 6, 2019, 6:19:42 AM PST, Rahul Reddy <
> rahulreddy1234@gmail.com> wrote:
>
>
> Hello,
>
> I see maximum memory usage alerts in my system.log couple of times in a
> day as INFO. So far I haven't seen any issue with db. Why those messages
> are logged in system.log do we have any impact for reads/writes with those
> warnings? And what nerd to be looked
>
> INFO [RMI TCP Connection(170917)-127.0.0.1] 2019-02-05 23:15:47,408
> NoSpamLogger.java:91 - Maximum memory usage reached (512.000MiB), cannot
> allocate chunk of 1.000MiB
>
> Thanks in advance
>
Re: Maximum memory usage
Posted by Rahul Reddy <ra...@gmail.com>.
No not running any nodetool commands. It happens 2 to 3 times a day
On Thu, Feb 7, 2019, 2:29 AM dinesh.joshi@yahoo.com.INVALID
<di...@yahoo.com.invalid> wrote:
> Are you running any nodetool commands during that period? IIRC, this is a
> log entry emitted by the BufferPool. It may be harm unless it's happening
> very often or logging a OOM.
>
> Dinesh
>
>
> On Wednesday, February 6, 2019, 6:19:42 AM PST, Rahul Reddy <
> rahulreddy1234@gmail.com> wrote:
>
>
> Hello,
>
> I see maximum memory usage alerts in my system.log couple of times in a
> day as INFO. So far I haven't seen any issue with db. Why those messages
> are logged in system.log do we have any impact for reads/writes with those
> warnings? And what nerd to be looked
>
> INFO [RMI TCP Connection(170917)-127.0.0.1] 2019-02-05 23:15:47,408
> NoSpamLogger.java:91 - Maximum memory usage reached (512.000MiB), cannot
> allocate chunk of 1.000MiB
>
> Thanks in advance
>
Re: Maximum memory usage
Posted by "dinesh.joshi@yahoo.com.INVALID" <di...@yahoo.com.INVALID>.
Are you running any nodetool commands during that period? IIRC, this is a log entry emitted by the BufferPool. It may be harm unless it's happening very often or logging a OOM.
Dinesh
On Wednesday, February 6, 2019, 6:19:42 AM PST, Rahul Reddy <ra...@gmail.com> wrote:
Hello,
I see maximum memory usage alerts in my system.log couple of times in a day as INFO. So far I haven't seen any issue with db. Why those messages are logged in system.log do we have any impact for reads/writes with those warnings? And what nerd to be looked
INFO [RMI TCP Connection(170917)-127.0.0.1] 2019-02-05 23:15:47,408 NoSpamLogger.java:91 - Maximum memory usage reached (512.000MiB), cannot allocate chunk of 1.000MiB
Thanks in advance
RE: Maximum memory usage
Posted by Kenneth Brotman <ke...@yahoo.com.INVALID>.
Can we the see “nodetool tablestats” for the biggest table as well.
From: Kenneth Brotman [mailto:kenbrotman@yahoo.com.INVALID]
Sent: Sunday, February 10, 2019 7:21 AM
To: user@cassandra.apache.org
Subject: RE: Maximum memory usage
Okay, that’s at the moment it was calculated. Still need to see histograms.
From: Rahul Reddy [mailto:rahulreddy1234@gmail.com]
Sent: Sunday, February 10, 2019 7:09 AM
To: user@cassandra.apache.org
Subject: Re: Maximum memory usage
Thanks Kenneth,
110mb is the biggest partition in our db
On Sun, Feb 10, 2019, 9:55 AM Kenneth Brotman <ke...@yahoo.com.invalid> wrote:
Rahul,
Those partitions are tiny. Could you give us the table histograms for the biggest tables.
Thanks,
Kenneth Brotman
From: Rahul Reddy [mailto:rahulreddy1234@gmail.com]
Sent: Sunday, February 10, 2019 6:43 AM
To: user@cassandra.apache.org
Subject: Re: Maximum memory usage
```Percentile SSTables Write Latency Read Latency Partition Size Cell Count
(micros) (micros) (bytes)
50% 1.00 24.60 219.34 258 4
75% 100 24.60 379.02 446 8
95% 1.00 35.43 379.02 924 17
98% 1.00 51.01 379.02 1109 24
99% 1.00 61.21 379.02 1331 29
Min 0.00 8.24 126.94 104 0
Max 1.00 152.32 379.02 8409007 152321
```
On Wed, Feb 6, 2019, 8:34 PM Kenneth Brotman <ke...@yahoo.com.invalid> wrote:
Can you give us the “nodetool tablehistograms”
Kenneth Brotman
From: Rahul Reddy [mailto:rahulreddy1234@gmail.com]
Sent: Wednesday, February 06, 2019 6:19 AM
To: user@cassandra.apache.org
Subject: Maximum memory usage
Hello,
I see maximum memory usage alerts in my system.log couple of times in a day as INFO. So far I haven't seen any issue with db. Why those messages are logged in system.log do we have any impact for reads/writes with those warnings? And what nerd to be looked
INFO [RMI TCP Connection(170917)-127.0.0.1] 2019-02-05 23:15:47,408 NoSpamLogger.java:91 - Maximum memory usage reached (512.000MiB), cannot allocate chunk of 1.000MiB
Thanks in advance
RE: Maximum memory usage
Posted by Kenneth Brotman <ke...@yahoo.com.INVALID>.
Okay, that’s at the moment it was calculated. Still need to see histograms.
From: Rahul Reddy [mailto:rahulreddy1234@gmail.com]
Sent: Sunday, February 10, 2019 7:09 AM
To: user@cassandra.apache.org
Subject: Re: Maximum memory usage
Thanks Kenneth,
110mb is the biggest partition in our db
On Sun, Feb 10, 2019, 9:55 AM Kenneth Brotman <ke...@yahoo.com.invalid> wrote:
Rahul,
Those partitions are tiny. Could you give us the table histograms for the biggest tables.
Thanks,
Kenneth Brotman
From: Rahul Reddy [mailto:rahulreddy1234@gmail.com]
Sent: Sunday, February 10, 2019 6:43 AM
To: user@cassandra.apache.org
Subject: Re: Maximum memory usage
```Percentile SSTables Write Latency Read Latency Partition Size Cell Count
(micros) (micros) (bytes)
50% 1.00 24.60 219.34 258 4
75% 100 24.60 379.02 446 8
95% 1.00 35.43 379.02 924 17
98% 1.00 51.01 379.02 1109 24
99% 1.00 61.21 379.02 1331 29
Min 0.00 8.24 126.94 104 0
Max 1.00 152.32 379.02 8409007 152321
```
On Wed, Feb 6, 2019, 8:34 PM Kenneth Brotman <ke...@yahoo.com.invalid> wrote:
Can you give us the “nodetool tablehistograms”
Kenneth Brotman
From: Rahul Reddy [mailto:rahulreddy1234@gmail.com]
Sent: Wednesday, February 06, 2019 6:19 AM
To: user@cassandra.apache.org
Subject: Maximum memory usage
Hello,
I see maximum memory usage alerts in my system.log couple of times in a day as INFO. So far I haven't seen any issue with db. Why those messages are logged in system.log do we have any impact for reads/writes with those warnings? And what nerd to be looked
INFO [RMI TCP Connection(170917)-127.0.0.1] 2019-02-05 23:15:47,408 NoSpamLogger.java:91 - Maximum memory usage reached (512.000MiB), cannot allocate chunk of 1.000MiB
Thanks in advance
Re: Maximum memory usage
Posted by Rahul Reddy <ra...@gmail.com>.
Thanks Kenneth,
110mb is the biggest partition in our db
On Sun, Feb 10, 2019, 9:55 AM Kenneth Brotman <ke...@yahoo.com.invalid>
wrote:
> Rahul,
>
>
>
> Those partitions are tiny. Could you give us the table histograms for the
> biggest tables.
>
>
>
> Thanks,
>
>
>
> Kenneth Brotman
>
>
>
> *From:* Rahul Reddy [mailto:rahulreddy1234@gmail.com]
> *Sent:* Sunday, February 10, 2019 6:43 AM
> *To:* user@cassandra.apache.org
> *Subject:* Re: Maximum memory usage
>
>
>
> ```Percentile SSTables Write Latency Read Latency Partition
> Size Cell Count
>
> (micros) (micros)
> (bytes)
>
> 50% 1.00 24.60 219.34
> 258 4
>
> 75% 1.00 24.60 379.02
> 446 8
>
> 95% 1.00 35.43 379.02
> 924 17
>
> 98% 1.00 51.01 379.02
> 1109 24
>
> 99% 1.00 61.21 379.02
> 1331 29
>
> Min 0.00 8.24 126.94
> 104 0
>
> Max 1.00 152.32 379.02
> 8409007 152321
>
> ```
>
>
>
> On Wed, Feb 6, 2019, 8:34 PM Kenneth Brotman <ke...@yahoo.com.invalid>
> wrote:
>
> Can you give us the “nodetool tablehistograms”
>
>
>
> Kenneth Brotman
>
>
>
> *From:* Rahul Reddy [mailto:rahulreddy1234@gmail.com]
> *Sent:* Wednesday, February 06, 2019 6:19 AM
> *To:* user@cassandra.apache.org
> *Subject:* Maximum memory usage
>
>
>
> Hello,
>
>
>
> I see maximum memory usage alerts in my system.log couple of times in a
> day as INFO. So far I haven't seen any issue with db. Why those messages
> are logged in system.log do we have any impact for reads/writes with those
> warnings? And what nerd to be looked
>
>
>
> INFO [RMI TCP Connection(170917)-127.0.0.1] 2019-02-05 23:15:47,408
> NoSpamLogger.java:91 - Maximum memory usage reached (512.000MiB), cannot
> allocate chunk of 1.000MiB
>
>
>
> Thanks in advance
>
>
RE: Maximum memory usage
Posted by Kenneth Brotman <ke...@yahoo.com.INVALID>.
Rahul,
Those partitions are tiny. Could you give us the table histograms for the biggest tables.
Thanks,
Kenneth Brotman
From: Rahul Reddy [mailto:rahulreddy1234@gmail.com]
Sent: Sunday, February 10, 2019 6:43 AM
To: user@cassandra.apache.org
Subject: Re: Maximum memory usage
```Percentile SSTables Write Latency Read Latency Partition Size Cell Count
(micros) (micros) (bytes)
50% 1.00 24.60 219.34 258 4
75% 1.00 24.60 379.02 446 8
95% 1.00 35.43 379.02 924 17
98% 1.00 51.01 379.02 1109 24
99% 1.00 61.21 379.02 1331 29
Min 0.00 8.24 126.94 104 0
Max 1.00 152.32 379.02 8409007 152321
```
On Wed, Feb 6, 2019, 8:34 PM Kenneth Brotman <ke...@yahoo.com.invalid> wrote:
Can you give us the “nodetool tablehistograms”
Kenneth Brotman
From: Rahul Reddy [mailto:rahulreddy1234@gmail.com]
Sent: Wednesday, February 06, 2019 6:19 AM
To: user@cassandra.apache.org
Subject: Maximum memory usage
Hello,
I see maximum memory usage alerts in my system.log couple of times in a day as INFO. So far I haven't seen any issue with db. Why those messages are logged in system.log do we have any impact for reads/writes with those warnings? And what nerd to be looked
INFO [RMI TCP Connection(170917)-127.0.0.1] 2019-02-05 23:15:47,408 NoSpamLogger.java:91 - Maximum memory usage reached (512.000MiB), cannot allocate chunk of 1.000MiB
Thanks in advance
Re: Maximum memory usage
Posted by Rahul Reddy <ra...@gmail.com>.
```Percentile SSTables Write Latency Read Latency Partition
Size Cell Count
(micros) (micros) (bytes)
50% 1.00 24.60 219.34 258
4
75% 1.00 24.60 379.02 446
8
95% 1.00 35.43 379.02 924
17
98% 1.00 51.01 379.02 1109
24
99% 1.00 61.21 379.02 1331
29
Min 0.00 8.24 126.94 104
0
Max 1.00 152.32 379.02 8409007
152321
```
On Wed, Feb 6, 2019, 8:34 PM Kenneth Brotman <ke...@yahoo.com.invalid>
wrote:
> Can you give us the “nodetool tablehistograms”
>
>
>
> Kenneth Brotman
>
>
>
> *From:* Rahul Reddy [mailto:rahulreddy1234@gmail.com]
> *Sent:* Wednesday, February 06, 2019 6:19 AM
> *To:* user@cassandra.apache.org
> *Subject:* Maximum memory usage
>
>
>
> Hello,
>
>
>
> I see maximum memory usage alerts in my system.log couple of times in a
> day as INFO. So far I haven't seen any issue with db. Why those messages
> are logged in system.log do we have any impact for reads/writes with those
> warnings? And what nerd to be looked
>
>
>
> INFO [RMI TCP Connection(170917)-127.0.0.1] 2019-02-05 23:15:47,408
> NoSpamLogger.java:91 - Maximum memory usage reached (512.000MiB), cannot
> allocate chunk of 1.000MiB
>
>
>
> Thanks in advance
>
RE: Maximum memory usage
Posted by Kenneth Brotman <ke...@yahoo.com.INVALID>.
Can you give us the “nodetool tablehistograms”
Kenneth Brotman
From: Rahul Reddy [mailto:rahulreddy1234@gmail.com]
Sent: Wednesday, February 06, 2019 6:19 AM
To: user@cassandra.apache.org
Subject: Maximum memory usage
Hello,
I see maximum memory usage alerts in my system.log couple of times in a day as INFO. So far I haven't seen any issue with db. Why those messages are logged in system.log do we have any impact for reads/writes with those warnings? And what nerd to be looked
INFO [RMI TCP Connection(170917)-127.0.0.1] 2019-02-05 23:15:47,408 NoSpamLogger.java:91 - Maximum memory usage reached (512.000MiB), cannot allocate chunk of 1.000MiB
Thanks in advance