You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Maciej Miklas <ma...@googlemail.com> on 2011/11/01 10:15:27 UTC
Cassandra 1.x and proper JNA setup
Hi all,
is there any documentation about proper JNA configuration?
I do not understand few things:
1) Does JNA use JVM heap settings?
2) Do I need to decrease max heap size while using JNA?
3) How do I limit RAM allocated by JNA?
4) Where can I see / monitor row cache size?
5) I've configured JNA just for test on my dev computer and so far
I've noticed serious performance issues (high cpu usage on heavy write
load), so I must be doing something wrong.... I've just copied JNA
jars into Cassandra/lib, without installing any native libs. This
should not work at all, right?
Thanks,
Maciej
Re: Cassandra 1.x and proper JNA setup
Posted by Maciej Miklas <ma...@googlemail.com>.
Super - thank you for help :)
On Thu, Nov 3, 2011 at 6:55 PM, Jonathan Ellis <jb...@gmail.com> wrote:
> Relying on that was always a terrible idea because you could easily
> OOM before it could help. There's no substitute for "don't make the
> caches too large" in the first place.
>
> We're working on https://issues.apache.org/jira/browse/CASSANDRA-3143
> to make cache sizing easier.
>
> On Thu, Nov 3, 2011 at 3:16 AM, Maciej Miklas <ma...@googlemail.com>
> wrote:
> > According to source code, JNA is being used to call malloc and free. In
> this
> > case each cached row will be serialized into RAM.
> > We must be really careful when defining cache size - to large size would
> > cause out of memory. Previous Cassandra releases has logic that would
> > decrease cache size if heap is low.
> > Currently each row will be serialized without any memory limit checks -
> > assuming that I understood it right.
> >
> > Those properties:
> > reduce_cache_sizes_at: 0.85
> > reduce_cache_capacity_to: 0.6
> > are not used anymore - at least not when JNA is enabled, witch is default
> > from Cassandra 1.0
> >
> >
> > On Wed, Nov 2, 2011 at 1:53 PM, Maciej Miklas <mac.miklas@googlemail.com
> >
> > wrote:
> >>
> >> I've just found, that JNA will be not used from 1.1 release -
> >> https://issues.apache.org/jira/browse/CASSANDRA-3271
> >> I would be also nice to know what was the reason for this decision.
> >>
> >> Regards,
> >> Maciej
> >>
> >> On Wed, Nov 2, 2011 at 1:34 PM, Viktor Jevdokimov
> >> <Vi...@adform.com> wrote:
> >>>
> >>> Up, also interested in answers to questions below.
> >>>
> >>>
> >>> Best regards/ Pagarbiai
> >>>
> >>> Viktor Jevdokimov
> >>> Senior Developer
> >>>
> >>> Email: Viktor.Jevdokimov@adform.com
> >>> Phone: +370 5 212 3063
> >>> Fax: +370 5 261 0453
> >>>
> >>> J. Jasinskio 16C,
> >>> LT-01112 Vilnius,
> >>> Lithuania
> >>>
> >>>
> >>>
> >>> Disclaimer: The information contained in this message and attachments
> is
> >>> intended solely for the attention and use of the named addressee and
> may be
> >>> confidential. If you are not the intended recipient, you are reminded
> that
> >>> the information remains the property of the sender. You must not use,
> >>> disclose, distribute, copy, print or rely on this e-mail. If you have
> >>> received this message in error, please contact the sender immediately
> and
> >>> irrevocably delete this message and any copies.-----Original
> Message-----
> >>> From: Maciej Miklas [mailto:mac.miklas@googlemail.com]
> >>> Sent: Tuesday, November 01, 2011 11:15
> >>> To: user@cassandra.apache.org
> >>> Subject: Cassandra 1.x and proper JNA setup
> >>>
> >>> Hi all,
> >>>
> >>> is there any documentation about proper JNA configuration?
> >>>
> >>> I do not understand few things:
> >>>
> >>> 1) Does JNA use JVM heap settings?
> >>>
> >>> 2) Do I need to decrease max heap size while using JNA?
> >>>
> >>> 3) How do I limit RAM allocated by JNA?
> >>>
> >>> 4) Where can I see / monitor row cache size?
> >>>
> >>> 5) I've configured JNA just for test on my dev computer and so far I've
> >>> noticed serious performance issues (high cpu usage on heavy write
> load), so
> >>> I must be doing something wrong.... I've just copied JNA jars into
> >>> Cassandra/lib, without installing any native libs. This should not
> work at
> >>> all, right?
> >>>
> >>> Thanks,
> >>> Maciej
> >>>
> >>
> >
> >
>
>
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of DataStax, the source for professional Cassandra support
> http://www.datastax.com
>
Re: Cassandra 1.x and proper JNA setup
Posted by Jonathan Ellis <jb...@gmail.com>.
Relying on that was always a terrible idea because you could easily
OOM before it could help. There's no substitute for "don't make the
caches too large" in the first place.
We're working on https://issues.apache.org/jira/browse/CASSANDRA-3143
to make cache sizing easier.
On Thu, Nov 3, 2011 at 3:16 AM, Maciej Miklas <ma...@googlemail.com> wrote:
> According to source code, JNA is being used to call malloc and free. In this
> case each cached row will be serialized into RAM.
> We must be really careful when defining cache size - to large size would
> cause out of memory. Previous Cassandra releases has logic that would
> decrease cache size if heap is low.
> Currently each row will be serialized without any memory limit checks -
> assuming that I understood it right.
>
> Those properties:
> reduce_cache_sizes_at: 0.85
> reduce_cache_capacity_to: 0.6
> are not used anymore - at least not when JNA is enabled, witch is default
> from Cassandra 1.0
>
>
> On Wed, Nov 2, 2011 at 1:53 PM, Maciej Miklas <ma...@googlemail.com>
> wrote:
>>
>> I've just found, that JNA will be not used from 1.1 release -
>> https://issues.apache.org/jira/browse/CASSANDRA-3271
>> I would be also nice to know what was the reason for this decision.
>>
>> Regards,
>> Maciej
>>
>> On Wed, Nov 2, 2011 at 1:34 PM, Viktor Jevdokimov
>> <Vi...@adform.com> wrote:
>>>
>>> Up, also interested in answers to questions below.
>>>
>>>
>>> Best regards/ Pagarbiai
>>>
>>> Viktor Jevdokimov
>>> Senior Developer
>>>
>>> Email: Viktor.Jevdokimov@adform.com
>>> Phone: +370 5 212 3063
>>> Fax: +370 5 261 0453
>>>
>>> J. Jasinskio 16C,
>>> LT-01112 Vilnius,
>>> Lithuania
>>>
>>>
>>>
>>> Disclaimer: The information contained in this message and attachments is
>>> intended solely for the attention and use of the named addressee and may be
>>> confidential. If you are not the intended recipient, you are reminded that
>>> the information remains the property of the sender. You must not use,
>>> disclose, distribute, copy, print or rely on this e-mail. If you have
>>> received this message in error, please contact the sender immediately and
>>> irrevocably delete this message and any copies.-----Original Message-----
>>> From: Maciej Miklas [mailto:mac.miklas@googlemail.com]
>>> Sent: Tuesday, November 01, 2011 11:15
>>> To: user@cassandra.apache.org
>>> Subject: Cassandra 1.x and proper JNA setup
>>>
>>> Hi all,
>>>
>>> is there any documentation about proper JNA configuration?
>>>
>>> I do not understand few things:
>>>
>>> 1) Does JNA use JVM heap settings?
>>>
>>> 2) Do I need to decrease max heap size while using JNA?
>>>
>>> 3) How do I limit RAM allocated by JNA?
>>>
>>> 4) Where can I see / monitor row cache size?
>>>
>>> 5) I've configured JNA just for test on my dev computer and so far I've
>>> noticed serious performance issues (high cpu usage on heavy write load), so
>>> I must be doing something wrong.... I've just copied JNA jars into
>>> Cassandra/lib, without installing any native libs. This should not work at
>>> all, right?
>>>
>>> Thanks,
>>> Maciej
>>>
>>
>
>
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com
Re: Cassandra 1.x and proper JNA setup
Posted by Maciej Miklas <ma...@googlemail.com>.
According to source code, JNA is being used to call malloc and free. In
this case each cached row will be serialized into RAM.
We must be really careful when defining cache size - to large size would
cause out of memory. Previous Cassandra releases has logic that would
decrease cache size if heap is low.
Currently each row will be serialized without any memory limit checks -
assuming that I understood it right.
Those properties:
reduce_cache_sizes_at: 0.85
reduce_cache_capacity_to: 0.6
are not used anymore - at least not when JNA is enabled, witch is default
from Cassandra 1.0
On Wed, Nov 2, 2011 at 1:53 PM, Maciej Miklas <ma...@googlemail.com>wrote:
> I've just found, that JNA will be not used from 1.1 release -
> https://issues.apache.org/jira/browse/CASSANDRA-3271
> I would be also nice to know what was the reason for this decision.
>
> Regards,
> Maciej
>
>
> On Wed, Nov 2, 2011 at 1:34 PM, Viktor Jevdokimov <
> Viktor.Jevdokimov@adform.com> wrote:
>
>> Up, also interested in answers to questions below.
>>
>>
>> Best regards/ Pagarbiai
>>
>> Viktor Jevdokimov
>> Senior Developer
>>
>> Email: Viktor.Jevdokimov@adform.com
>> Phone: +370 5 212 3063
>> Fax: +370 5 261 0453
>>
>> J. Jasinskio 16C,
>> LT-01112 Vilnius,
>> Lithuania
>>
>>
>>
>> Disclaimer: The information contained in this message and attachments is
>> intended solely for the attention and use of the named addressee and may be
>> confidential. If you are not the intended recipient, you are reminded that
>> the information remains the property of the sender. You must not use,
>> disclose, distribute, copy, print or rely on this e-mail. If you have
>> received this message in error, please contact the sender immediately and
>> irrevocably delete this message and any copies.-----Original Message-----
>> From: Maciej Miklas [mailto:mac.miklas@googlemail.com]
>> Sent: Tuesday, November 01, 2011 11:15
>> To: user@cassandra.apache.org
>> Subject: Cassandra 1.x and proper JNA setup
>>
>> Hi all,
>>
>> is there any documentation about proper JNA configuration?
>>
>> I do not understand few things:
>>
>> 1) Does JNA use JVM heap settings?
>>
>> 2) Do I need to decrease max heap size while using JNA?
>>
>> 3) How do I limit RAM allocated by JNA?
>>
>> 4) Where can I see / monitor row cache size?
>>
>> 5) I've configured JNA just for test on my dev computer and so far I've
>> noticed serious performance issues (high cpu usage on heavy write load), so
>> I must be doing something wrong.... I've just copied JNA jars into
>> Cassandra/lib, without installing any native libs. This should not work at
>> all, right?
>>
>> Thanks,
>> Maciej
>>
>>
>
Re: Cassandra 1.x and proper JNA setup
Posted by Maciej Miklas <ma...@googlemail.com>.
I've just found, that JNA will be not used from 1.1 release -
https://issues.apache.org/jira/browse/CASSANDRA-3271
I would be also nice to know what was the reason for this decision.
Regards,
Maciej
On Wed, Nov 2, 2011 at 1:34 PM, Viktor Jevdokimov <
Viktor.Jevdokimov@adform.com> wrote:
> Up, also interested in answers to questions below.
>
>
> Best regards/ Pagarbiai
>
> Viktor Jevdokimov
> Senior Developer
>
> Email: Viktor.Jevdokimov@adform.com
> Phone: +370 5 212 3063
> Fax: +370 5 261 0453
>
> J. Jasinskio 16C,
> LT-01112 Vilnius,
> Lithuania
>
>
>
> Disclaimer: The information contained in this message and attachments is
> intended solely for the attention and use of the named addressee and may be
> confidential. If you are not the intended recipient, you are reminded that
> the information remains the property of the sender. You must not use,
> disclose, distribute, copy, print or rely on this e-mail. If you have
> received this message in error, please contact the sender immediately and
> irrevocably delete this message and any copies.-----Original Message-----
> From: Maciej Miklas [mailto:mac.miklas@googlemail.com]
> Sent: Tuesday, November 01, 2011 11:15
> To: user@cassandra.apache.org
> Subject: Cassandra 1.x and proper JNA setup
>
> Hi all,
>
> is there any documentation about proper JNA configuration?
>
> I do not understand few things:
>
> 1) Does JNA use JVM heap settings?
>
> 2) Do I need to decrease max heap size while using JNA?
>
> 3) How do I limit RAM allocated by JNA?
>
> 4) Where can I see / monitor row cache size?
>
> 5) I've configured JNA just for test on my dev computer and so far I've
> noticed serious performance issues (high cpu usage on heavy write load), so
> I must be doing something wrong.... I've just copied JNA jars into
> Cassandra/lib, without installing any native libs. This should not work at
> all, right?
>
> Thanks,
> Maciej
>
>
Re: unsubscribe
Posted by Eric Evans <ee...@acunu.com>.
On Wed, Nov 2, 2011 at 7:44 AM, Antonio VA Hilario <aa...@gmail.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> - -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> unsubscribe
http://threepanelsoul.com/2008/12/02/on-sea-lions/
--
Eric Evans
Acunu | http://www.acunu.com | @acunu
unsubscribe
Posted by Antonio VA Hilario <aa...@gmail.com>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
- -----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
unsubscribe
- -----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
iQIcBAEBAgAGBQJOsTpDAAoJEMDwM2gyrmVA86wQAJ2J61KDXMb8zxIKLq9CV8vS
ZKykA2pHHiJ8Jja2Ia7EH5tAJvkpYhsuJeLM4MBb3CK0O5J9wjluygerWPaWx1SW
pKFRH0Ah7lvv1MO0KP37Xn6HwN8rN4hau6WoF58YLVu1Oe7vDq74Bry+PH2QUG0y
FdLxfJGGH0RrV4nk4nV64FaNa6xAJziMgZTnw9h2zy5CSlLfE+IurrFhimjFGKta
8rUkv9ucaG25TKf0hUCVS0oBjr37ChnPVNsM3m7ri6N0OFQi6ciSL38i4r0HW+DP
fzRlCp7r7m9huNIlZdfFFS4wE9ipKNIb/i9+4BXF+F1fUmzq0UzEeNUzOXjZCLLA
ogH7ira8l662aOmsqhd071lZaXPPAcNYG2z+mY22XCUbzNjfAsc8vBViEiBR8/dd
Qkyf/xbaEZHuhnQFDNCtp8z3uixQMGaYZv1he1+rWfmrzbuxk4yssIJOuDvcSGBp
cupw1/LzivDgQALLYyymPX0w2UIOTjSLzEMW9Ij6QK1bQe0g0O6Jo5wjmyXsZbu1
BeHwRgyQg3f5o3I2VMOYEJ5Q/Z1LYvrN2q8scs8OH44c0tlEpP0yGm41ak33Vezq
4/in24Dg/vU4TN+7yIEPffqXFZUlkX3ZTdolQ7xC5wI3rg1oAh2USppDGX25vlgC
8isFuZn0kY9yETJr7ufR
=Y+8x
- -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
iQIcBAEBAgAGBQJOsTsQAAoJEMDwM2gyrmVADwcP/iuTUsUiTyzHjPfRiI165iX9
xjekg/xcEC+4rHhjFwKsvXuR1JUHALLB2rpdLR6cMqvNb/Udp0Y0Y7rS/gTJD7yC
ApBZoxJ90MUD1AD6TadFTe1WhD4B+aQCs36kKX4YoPBTaaTbkzErgC77JZlAqbXP
4rSsnZt/zn/85KyopH1RXTyXFZFtxdhmsxIvJIfuUmWz8BXR25L6lauyv5KUbef2
0wdRz1f4gasvnk3xthJo7+G0UxZirt95rFdctD3+wgy1jlCdnG5Ft7vlRfxDX5FP
rnSl9Vnr1akP6qt260ZxTvG0YtmH6Z4/bVtHDhcerbYbUQ21+yAtjrtCzhmGnru7
3tCgnDCoPyjxMCpVbgCc4GUVu8ei4Zkywc/xsRLlHqGQM7tta5chJOy/QKHuv/q0
b2ohPcFlzcBmQmSjFY5NM2ddyjP5ERvgkfRCMbSF+1MMz1WCJ3ijWK8ahFGcw7T/
7On7k5OX24uSkb5dqC26OToEEenODgJ6sAgPM94qbG5EfPAwYF7kHLGGXGCHLMQc
VLt0oMHhhkaEskVPjMMwZa0l2WKz9CSK4f4694tzHU6eSH0v0tMZbg2493vnkmXw
/F65Gm1RHwbATXkYSwxNXOPuOO72Oj76OtUVmT72cvtFw+dbZB6ZWjp7fyli7Eg4
gpQ/GF3DZOp25cxMY8Ra
=b3Y5
-----END PGP SIGNATURE-----
RE: Cassandra 1.x and proper JNA setup
Posted by Viktor Jevdokimov <Vi...@adform.com>.
Up, also interested in answers to questions below.
Best regards/ Pagarbiai
Viktor Jevdokimov
Senior Developer
Email: Viktor.Jevdokimov@adform.com
Phone: +370 5 212 3063
Fax: +370 5 261 0453
J. Jasinskio 16C,
LT-01112 Vilnius,
Lithuania
Disclaimer: The information contained in this message and attachments is intended solely for the attention and use of the named addressee and may be confidential. If you are not the intended recipient, you are reminded that the information remains the property of the sender. You must not use, disclose, distribute, copy, print or rely on this e-mail. If you have received this message in error, please contact the sender immediately and irrevocably delete this message and any copies.-----Original Message-----
From: Maciej Miklas [mailto:mac.miklas@googlemail.com]
Sent: Tuesday, November 01, 2011 11:15
To: user@cassandra.apache.org
Subject: Cassandra 1.x and proper JNA setup
Hi all,
is there any documentation about proper JNA configuration?
I do not understand few things:
1) Does JNA use JVM heap settings?
2) Do I need to decrease max heap size while using JNA?
3) How do I limit RAM allocated by JNA?
4) Where can I see / monitor row cache size?
5) I've configured JNA just for test on my dev computer and so far I've noticed serious performance issues (high cpu usage on heavy write load), so I must be doing something wrong.... I've just copied JNA jars into Cassandra/lib, without installing any native libs. This should not work at all, right?
Thanks,
Maciej