You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by yann Blazart <ya...@gmail.com> on 2019/05/25 07:53:35 UTC
How to know memory used by a cache or a set
Hello !
Because I have multiple process that compute files in ignite H2 database,
I really need to know how memory have been used by caches and sets,
but the persistence is not activated (performance) and when I try to use
the MxBean for CacheGroup, it send me back a 0.
Is there any solution ??
Thanks.
Re: How to know memory used by a cache or a set
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
> Or do we have 79Gb allocated AND my data only take 44% of space in it ?
This is correct.
"comm" is amount of memory allocated by underlying operating system, as far
as my understanding goes.
Regards,
--
Ilya Kasnacheev
ср, 26 июн. 2019 г. в 12:14, yann.blazart@externe.bnpparibas.com <
yann.blazart@externe.bnpparibas.com>:
> Hello, can you help me o understand the real mean of this trace ?
>
> Off-heap [used=79492MB, free=56.92%, comm=86642MB]
> ^-- sysMemPlc region [used=0MB, free=99.13%, comm=40MB]
> ^-- Gondor region [used=79491MB, free=56.87%, comm=86562MB]
>
>
> My region is called Gondor, is "used" is the amount of memory really used
> in this node ? In this case what's the meaning of "free" and "comm" ?
>
> Or do we have 79Gb allocated AND my data only take 44% of space in it ?
>
> Thanks, and regards.
>
>
> Le 26 juin 2019 à 01:28, Denis Magda <dm...@apache.org> a écrit :
>
> Looks like a mess.
>
> Alex Goncharuk, Nikolay Izhikov, considering the latest changes and new
> metrics & tracing framework, how would they one get cache/table memory size
> via a simple metric? There should be a way to make it workable without
> hacks like cachegroups, etc.
>
>
> -
> Denis
>
>
> On Wed, Jun 19, 2019 at 3:29 AM Alex Plehanov <pl...@gmail.com>
> wrote:
>
>> Denis,
>>
>> Documentation for memory usage calculation covers another case (memory
>> usage by the node).
>> There is no ability (AFAIK) in released Ignite versions to calculate
>> memory used by a cache or cache group when persistence is disabled.
>> Dedicated data region can be used for some of the caches in some cases
>> and metrics can be collected for this data region, but when the cache is
>> destroyed (or data is deleted) memory is not deallocated, it's going to
>> reuse list.
>> There is a new metric implemented DataRegionMetrics#getTotalUsedPages
>> (count of allocated pages minus count of pages in the reuse list) which
>> will partially help to solve Yann's problem, but this metric will be
>> available only in the next Ignite release.
>> Also, as a temporary workaround, some internal API can be used to get a
>> count of pages in the reuse list and calculate total used pages by data
>> region manually.
>>
>> ср, 19 июн. 2019 г. в 07:59, Denis Magda <dm...@gridgain.com>:
>>
>>> + dev list
>>>
>>> Ignite developers,
>>>
>>> Seems that the present solution for memory calculation doesn't work
>>> (check
>>> the thread):
>>>
>>> https://apacheignite.readme.io/v2.5/docs/memory-metrics#section-memory-usage-calculation
>>> <https://apacheignite.readme.io/docs/cache-metrics>
>>>
>>> Was it really broken?
>>>
>>> --
>>> Denis Magda
>>>
>>>
>>> On Thu, Jun 13, 2019 at 9:45 AM yann Blazart <ya...@gmail.com>
>>> wrote:
>>>
>>> > Ok, but I can't create dynamically a data region ? Because each time I
>>> > receive a new file to process, I create a cachegroup to handle it,
>>> then I
>>> > clean everything.
>>> >
>>> > Le jeu. 13 juin 2019 à 13:28, Alex Plehanov <pl...@gmail.com>
>>> a
>>> > écrit :
>>> >
>>> >> Hello,
>>> >>
>>> >> It's a known issue [1]. Now you can get cache group size via JMX only
>>> if
>>> >> persistence is used.
>>> >> If persistence is not used you can get allocated size only for data
>>> >> region (but you can have more then one data region and assign cache
>>> groups
>>> >> to data regions any way you want)
>>> >>
>>> >> [1] : https://issues.apache.org/jira/browse/IGNITE-8517
>>> >>
>>> >> ср, 12 июн. 2019 г. в 16:00, yann.blazart@externe.bnpparibas.com <
>>> >> yann.blazart@externe.bnpparibas.com>:
>>> >>
>>> >>> Hello, I'm back.
>>> >>>
>>> >>> Well, I need to get memory used by each execution of my process, so I
>>> >>> put all involved caches into the same cacheGroup.
>>> >>>
>>> >>> If I use the CacheGroupMetricsBean, the size gave to me is 0 !
>>> >>> If I enable persistence on DataRegion, I get size, but I don't want
>>> to
>>> >>> use the persistence enabled.
>>> >>>
>>> >>> Is it a bug ?
>>> >>>
>>> >>> regards.
>>> >>>
>>> >>> > Le 27 mai 2019 à 11:09, ibelyakov <ig...@gmail.com> a
>>> écrit
>>> >>> :
>>> >>> >
>>> >>> > Hi,
>>> >>> >
>>> >>> > Did you turn on cache metrics for your data region?
>>> >>> >
>>> >>> > To turn the metrics on, use one of the following approaches:
>>> >>> > 1. Set DataRegionConfiguration.setMetricsEnabled(true) for every
>>> >>> region you
>>> >>> > want to collect the metrics for.
>>> >>> > 2. Use the DataRegionMetricsMXBean.enableMetrics() method exposed
>>> by a
>>> >>> > special JMX bean.
>>> >>> >
>>> >>> > More information regarding cache metrics available here:
>>> >>> > https://apacheignite.readme.io/docs/cache-metrics
>>> >>> >
>>> >>> > Regards,
>>> >>> > Igor
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> > --
>>> >>> > Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>> >>>
>>> >>> This message and any attachments (the "message") is
>>> >>> intended solely for the intended addressees and is confidential.
>>> >>> If you receive this message in error,or are not the intended
>>> >>> recipient(s),
>>> >>> please delete it and any copies from your systems and immediately
>>> notify
>>> >>> the sender. Any unauthorized view, use that does not comply with its
>>> >>> purpose,
>>> >>> dissemination or disclosure, either whole or partial, is prohibited.
>>> >>> Since the internet
>>> >>> cannot guarantee the integrity of this message which may not be
>>> >>> reliable, BNP PARIBAS
>>> >>> (and its subsidiaries) shall not be liable for the message if
>>> modified,
>>> >>> changed or falsified.
>>> >>> Do not print this message unless it is necessary, consider the
>>> >>> environment.
>>> >>>
>>> >>>
>>> >>>
>>> ----------------------------------------------------------------------------------------------------------------------------------
>>> >>>
>>> >>> Ce message et toutes les pieces jointes (ci-apres le "message")
>>> >>> sont etablis a l'intention exclusive de ses destinataires et sont
>>> >>> confidentiels.
>>> >>> Si vous recevez ce message par erreur ou s'il ne vous est pas
>>> destine,
>>> >>> merci de le detruire ainsi que toute copie de votre systeme et d'en
>>> >>> avertir
>>> >>> immediatement l'expediteur. Toute lecture non autorisee, toute
>>> >>> utilisation de
>>> >>> ce message qui n'est pas conforme a sa destination, toute diffusion
>>> ou
>>> >>> toute
>>> >>> publication, totale ou partielle, est interdite. L'Internet ne
>>> >>> permettant pas d'assurer
>>> >>> l'integrite de ce message electronique susceptible d'alteration, BNP
>>> >>> Paribas
>>> >>> (et ses filiales) decline(nt) toute responsabilite au titre de ce
>>> >>> message dans l'hypothese
>>> >>> ou il aurait ete modifie, deforme ou falsifie.
>>> >>> N'imprimez ce message que si necessaire, pensez a l'environnement.
>>> >>>
>>> >>
>>>
>>
>
Re: How to know memory used by a cache or a set
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
> Or do we have 79Gb allocated AND my data only take 44% of space in it ?
This is correct.
"comm" is amount of memory allocated by underlying operating system, as far
as my understanding goes.
Regards,
--
Ilya Kasnacheev
ср, 26 июн. 2019 г. в 12:14, yann.blazart@externe.bnpparibas.com <
yann.blazart@externe.bnpparibas.com>:
> Hello, can you help me o understand the real mean of this trace ?
>
> Off-heap [used=79492MB, free=56.92%, comm=86642MB]
> ^-- sysMemPlc region [used=0MB, free=99.13%, comm=40MB]
> ^-- Gondor region [used=79491MB, free=56.87%, comm=86562MB]
>
>
> My region is called Gondor, is "used" is the amount of memory really used
> in this node ? In this case what's the meaning of "free" and "comm" ?
>
> Or do we have 79Gb allocated AND my data only take 44% of space in it ?
>
> Thanks, and regards.
>
>
> Le 26 juin 2019 à 01:28, Denis Magda <dm...@apache.org> a écrit :
>
> Looks like a mess.
>
> Alex Goncharuk, Nikolay Izhikov, considering the latest changes and new
> metrics & tracing framework, how would they one get cache/table memory size
> via a simple metric? There should be a way to make it workable without
> hacks like cachegroups, etc.
>
>
> -
> Denis
>
>
> On Wed, Jun 19, 2019 at 3:29 AM Alex Plehanov <pl...@gmail.com>
> wrote:
>
>> Denis,
>>
>> Documentation for memory usage calculation covers another case (memory
>> usage by the node).
>> There is no ability (AFAIK) in released Ignite versions to calculate
>> memory used by a cache or cache group when persistence is disabled.
>> Dedicated data region can be used for some of the caches in some cases
>> and metrics can be collected for this data region, but when the cache is
>> destroyed (or data is deleted) memory is not deallocated, it's going to
>> reuse list.
>> There is a new metric implemented DataRegionMetrics#getTotalUsedPages
>> (count of allocated pages minus count of pages in the reuse list) which
>> will partially help to solve Yann's problem, but this metric will be
>> available only in the next Ignite release.
>> Also, as a temporary workaround, some internal API can be used to get a
>> count of pages in the reuse list and calculate total used pages by data
>> region manually.
>>
>> ср, 19 июн. 2019 г. в 07:59, Denis Magda <dm...@gridgain.com>:
>>
>>> + dev list
>>>
>>> Ignite developers,
>>>
>>> Seems that the present solution for memory calculation doesn't work
>>> (check
>>> the thread):
>>>
>>> https://apacheignite.readme.io/v2.5/docs/memory-metrics#section-memory-usage-calculation
>>> <https://apacheignite.readme.io/docs/cache-metrics>
>>>
>>> Was it really broken?
>>>
>>> --
>>> Denis Magda
>>>
>>>
>>> On Thu, Jun 13, 2019 at 9:45 AM yann Blazart <ya...@gmail.com>
>>> wrote:
>>>
>>> > Ok, but I can't create dynamically a data region ? Because each time I
>>> > receive a new file to process, I create a cachegroup to handle it,
>>> then I
>>> > clean everything.
>>> >
>>> > Le jeu. 13 juin 2019 à 13:28, Alex Plehanov <pl...@gmail.com>
>>> a
>>> > écrit :
>>> >
>>> >> Hello,
>>> >>
>>> >> It's a known issue [1]. Now you can get cache group size via JMX only
>>> if
>>> >> persistence is used.
>>> >> If persistence is not used you can get allocated size only for data
>>> >> region (but you can have more then one data region and assign cache
>>> groups
>>> >> to data regions any way you want)
>>> >>
>>> >> [1] : https://issues.apache.org/jira/browse/IGNITE-8517
>>> >>
>>> >> ср, 12 июн. 2019 г. в 16:00, yann.blazart@externe.bnpparibas.com <
>>> >> yann.blazart@externe.bnpparibas.com>:
>>> >>
>>> >>> Hello, I'm back.
>>> >>>
>>> >>> Well, I need to get memory used by each execution of my process, so I
>>> >>> put all involved caches into the same cacheGroup.
>>> >>>
>>> >>> If I use the CacheGroupMetricsBean, the size gave to me is 0 !
>>> >>> If I enable persistence on DataRegion, I get size, but I don't want
>>> to
>>> >>> use the persistence enabled.
>>> >>>
>>> >>> Is it a bug ?
>>> >>>
>>> >>> regards.
>>> >>>
>>> >>> > Le 27 mai 2019 à 11:09, ibelyakov <ig...@gmail.com> a
>>> écrit
>>> >>> :
>>> >>> >
>>> >>> > Hi,
>>> >>> >
>>> >>> > Did you turn on cache metrics for your data region?
>>> >>> >
>>> >>> > To turn the metrics on, use one of the following approaches:
>>> >>> > 1. Set DataRegionConfiguration.setMetricsEnabled(true) for every
>>> >>> region you
>>> >>> > want to collect the metrics for.
>>> >>> > 2. Use the DataRegionMetricsMXBean.enableMetrics() method exposed
>>> by a
>>> >>> > special JMX bean.
>>> >>> >
>>> >>> > More information regarding cache metrics available here:
>>> >>> > https://apacheignite.readme.io/docs/cache-metrics
>>> >>> >
>>> >>> > Regards,
>>> >>> > Igor
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> > --
>>> >>> > Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>> >>>
>>> >>> This message and any attachments (the "message") is
>>> >>> intended solely for the intended addressees and is confidential.
>>> >>> If you receive this message in error,or are not the intended
>>> >>> recipient(s),
>>> >>> please delete it and any copies from your systems and immediately
>>> notify
>>> >>> the sender. Any unauthorized view, use that does not comply with its
>>> >>> purpose,
>>> >>> dissemination or disclosure, either whole or partial, is prohibited.
>>> >>> Since the internet
>>> >>> cannot guarantee the integrity of this message which may not be
>>> >>> reliable, BNP PARIBAS
>>> >>> (and its subsidiaries) shall not be liable for the message if
>>> modified,
>>> >>> changed or falsified.
>>> >>> Do not print this message unless it is necessary, consider the
>>> >>> environment.
>>> >>>
>>> >>>
>>> >>>
>>> ----------------------------------------------------------------------------------------------------------------------------------
>>> >>>
>>> >>> Ce message et toutes les pieces jointes (ci-apres le "message")
>>> >>> sont etablis a l'intention exclusive de ses destinataires et sont
>>> >>> confidentiels.
>>> >>> Si vous recevez ce message par erreur ou s'il ne vous est pas
>>> destine,
>>> >>> merci de le detruire ainsi que toute copie de votre systeme et d'en
>>> >>> avertir
>>> >>> immediatement l'expediteur. Toute lecture non autorisee, toute
>>> >>> utilisation de
>>> >>> ce message qui n'est pas conforme a sa destination, toute diffusion
>>> ou
>>> >>> toute
>>> >>> publication, totale ou partielle, est interdite. L'Internet ne
>>> >>> permettant pas d'assurer
>>> >>> l'integrite de ce message electronique susceptible d'alteration, BNP
>>> >>> Paribas
>>> >>> (et ses filiales) decline(nt) toute responsabilite au titre de ce
>>> >>> message dans l'hypothese
>>> >>> ou il aurait ete modifie, deforme ou falsifie.
>>> >>> N'imprimez ce message que si necessaire, pensez a l'environnement.
>>> >>>
>>> >>
>>>
>>
>
Re: How to know memory used by a cache or a set
Posted by "yann.blazart@externe.bnpparibas.com" <ya...@externe.bnpparibas.com>.
Hello, can you help me o understand the real mean of this trace ?
Off-heap [used=79492MB, free=56.92%, comm=86642MB]
^-- sysMemPlc region [used=0MB, free=99.13%, comm=40MB]
^-- Gondor region [used=79491MB, free=56.87%, comm=86562MB]
My region is called Gondor, is "used" is the amount of memory really used in this node ? In this case what's the meaning of "free" and "comm" ?
Or do we have 79Gb allocated AND my data only take 44% of space in it ?
Thanks, and regards.
Le 26 juin 2019 à 01:28, Denis Magda <dm...@apache.org>> a écrit :
Looks like a mess.
Alex Goncharuk, Nikolay Izhikov, considering the latest changes and new metrics & tracing framework, how would they one get cache/table memory size via a simple metric? There should be a way to make it workable without hacks like cachegroups, etc.
-
Denis
On Wed, Jun 19, 2019 at 3:29 AM Alex Plehanov <pl...@gmail.com>> wrote:
Denis,
Documentation for memory usage calculation covers another case (memory usage by the node).
There is no ability (AFAIK) in released Ignite versions to calculate memory used by a cache or cache group when persistence is disabled.
Dedicated data region can be used for some of the caches in some cases and metrics can be collected for this data region, but when the cache is destroyed (or data is deleted) memory is not deallocated, it's going to reuse list.
There is a new metric implemented DataRegionMetrics#getTotalUsedPages (count of allocated pages minus count of pages in the reuse list) which will partially help to solve Yann's problem, but this metric will be available only in the next Ignite release.
Also, as a temporary workaround, some internal API can be used to get a count of pages in the reuse list and calculate total used pages by data region manually.
ср, 19 июн. 2019 г. в 07:59, Denis Magda <dm...@gridgain.com>>:
+ dev list
Ignite developers,
Seems that the present solution for memory calculation doesn't work (check
the thread):
https://apacheignite.readme.io/v2.5/docs/memory-metrics#section-memory-usage-calculation
<https://apacheignite.readme.io/docs/cache-metrics>
Was it really broken?
--
Denis Magda
On Thu, Jun 13, 2019 at 9:45 AM yann Blazart <ya...@gmail.com>> wrote:
> Ok, but I can't create dynamically a data region ? Because each time I
> receive a new file to process, I create a cachegroup to handle it, then I
> clean everything.
>
> Le jeu. 13 juin 2019 à 13:28, Alex Plehanov <pl...@gmail.com>> a
> écrit :
>
>> Hello,
>>
>> It's a known issue [1]. Now you can get cache group size via JMX only if
>> persistence is used.
>> If persistence is not used you can get allocated size only for data
>> region (but you can have more then one data region and assign cache groups
>> to data regions any way you want)
>>
>> [1] : https://issues.apache.org/jira/browse/IGNITE-8517
>>
>> ср, 12 июн. 2019 г. в 16:00, yann.blazart@externe.bnpparibas.com<ma...@externe.bnpparibas.com> <
>> yann.blazart@externe.bnpparibas.com<ma...@externe.bnpparibas.com>>:
>>
>>> Hello, I'm back.
>>>
>>> Well, I need to get memory used by each execution of my process, so I
>>> put all involved caches into the same cacheGroup.
>>>
>>> If I use the CacheGroupMetricsBean, the size gave to me is 0 !
>>> If I enable persistence on DataRegion, I get size, but I don't want to
>>> use the persistence enabled.
>>>
>>> Is it a bug ?
>>>
>>> regards.
>>>
>>> > Le 27 mai 2019 à 11:09, ibelyakov <ig...@gmail.com>> a écrit
>>> :
>>> >
>>> > Hi,
>>> >
>>> > Did you turn on cache metrics for your data region?
>>> >
>>> > To turn the metrics on, use one of the following approaches:
>>> > 1. Set DataRegionConfiguration.setMetricsEnabled(true) for every
>>> region you
>>> > want to collect the metrics for.
>>> > 2. Use the DataRegionMetricsMXBean.enableMetrics() method exposed by a
>>> > special JMX bean.
>>> >
>>> > More information regarding cache metrics available here:
>>> > https://apacheignite.readme.io/docs/cache-metrics
>>> >
>>> > Regards,
>>> > Igor
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>
>>> This message and any attachments (the "message") is
>>> intended solely for the intended addressees and is confidential.
>>> If you receive this message in error,or are not the intended
>>> recipient(s),
>>> please delete it and any copies from your systems and immediately notify
>>> the sender. Any unauthorized view, use that does not comply with its
>>> purpose,
>>> dissemination or disclosure, either whole or partial, is prohibited.
>>> Since the internet
>>> cannot guarantee the integrity of this message which may not be
>>> reliable, BNP PARIBAS
>>> (and its subsidiaries) shall not be liable for the message if modified,
>>> changed or falsified.
>>> Do not print this message unless it is necessary, consider the
>>> environment.
>>>
>>>
>>> ----------------------------------------------------------------------------------------------------------------------------------
>>>
>>> Ce message et toutes les pieces jointes (ci-apres le "message")
>>> sont etablis a l'intention exclusive de ses destinataires et sont
>>> confidentiels.
>>> Si vous recevez ce message par erreur ou s'il ne vous est pas destine,
>>> merci de le detruire ainsi que toute copie de votre systeme et d'en
>>> avertir
>>> immediatement l'expediteur. Toute lecture non autorisee, toute
>>> utilisation de
>>> ce message qui n'est pas conforme a sa destination, toute diffusion ou
>>> toute
>>> publication, totale ou partielle, est interdite. L'Internet ne
>>> permettant pas d'assurer
>>> l'integrite de ce message electronique susceptible d'alteration, BNP
>>> Paribas
>>> (et ses filiales) decline(nt) toute responsabilite au titre de ce
>>> message dans l'hypothese
>>> ou il aurait ete modifie, deforme ou falsifie.
>>> N'imprimez ce message que si necessaire, pensez a l'environnement.
>>>
>>
Re: How to know memory used by a cache or a set
Posted by "yann.blazart@externe.bnpparibas.com" <ya...@externe.bnpparibas.com>.
Hello, can you help me o understand the real mean of this trace ?
Off-heap [used=79492MB, free=56.92%, comm=86642MB]
^-- sysMemPlc region [used=0MB, free=99.13%, comm=40MB]
^-- Gondor region [used=79491MB, free=56.87%, comm=86562MB]
My region is called Gondor, is "used" is the amount of memory really used in this node ? In this case what's the meaning of "free" and "comm" ?
Or do we have 79Gb allocated AND my data only take 44% of space in it ?
Thanks, and regards.
Le 26 juin 2019 à 01:28, Denis Magda <dm...@apache.org>> a écrit :
Looks like a mess.
Alex Goncharuk, Nikolay Izhikov, considering the latest changes and new metrics & tracing framework, how would they one get cache/table memory size via a simple metric? There should be a way to make it workable without hacks like cachegroups, etc.
-
Denis
On Wed, Jun 19, 2019 at 3:29 AM Alex Plehanov <pl...@gmail.com>> wrote:
Denis,
Documentation for memory usage calculation covers another case (memory usage by the node).
There is no ability (AFAIK) in released Ignite versions to calculate memory used by a cache or cache group when persistence is disabled.
Dedicated data region can be used for some of the caches in some cases and metrics can be collected for this data region, but when the cache is destroyed (or data is deleted) memory is not deallocated, it's going to reuse list.
There is a new metric implemented DataRegionMetrics#getTotalUsedPages (count of allocated pages minus count of pages in the reuse list) which will partially help to solve Yann's problem, but this metric will be available only in the next Ignite release.
Also, as a temporary workaround, some internal API can be used to get a count of pages in the reuse list and calculate total used pages by data region manually.
ср, 19 июн. 2019 г. в 07:59, Denis Magda <dm...@gridgain.com>>:
+ dev list
Ignite developers,
Seems that the present solution for memory calculation doesn't work (check
the thread):
https://apacheignite.readme.io/v2.5/docs/memory-metrics#section-memory-usage-calculation
<https://apacheignite.readme.io/docs/cache-metrics>
Was it really broken?
--
Denis Magda
On Thu, Jun 13, 2019 at 9:45 AM yann Blazart <ya...@gmail.com>> wrote:
> Ok, but I can't create dynamically a data region ? Because each time I
> receive a new file to process, I create a cachegroup to handle it, then I
> clean everything.
>
> Le jeu. 13 juin 2019 à 13:28, Alex Plehanov <pl...@gmail.com>> a
> écrit :
>
>> Hello,
>>
>> It's a known issue [1]. Now you can get cache group size via JMX only if
>> persistence is used.
>> If persistence is not used you can get allocated size only for data
>> region (but you can have more then one data region and assign cache groups
>> to data regions any way you want)
>>
>> [1] : https://issues.apache.org/jira/browse/IGNITE-8517
>>
>> ср, 12 июн. 2019 г. в 16:00, yann.blazart@externe.bnpparibas.com<ma...@externe.bnpparibas.com> <
>> yann.blazart@externe.bnpparibas.com<ma...@externe.bnpparibas.com>>:
>>
>>> Hello, I'm back.
>>>
>>> Well, I need to get memory used by each execution of my process, so I
>>> put all involved caches into the same cacheGroup.
>>>
>>> If I use the CacheGroupMetricsBean, the size gave to me is 0 !
>>> If I enable persistence on DataRegion, I get size, but I don't want to
>>> use the persistence enabled.
>>>
>>> Is it a bug ?
>>>
>>> regards.
>>>
>>> > Le 27 mai 2019 à 11:09, ibelyakov <ig...@gmail.com>> a écrit
>>> :
>>> >
>>> > Hi,
>>> >
>>> > Did you turn on cache metrics for your data region?
>>> >
>>> > To turn the metrics on, use one of the following approaches:
>>> > 1. Set DataRegionConfiguration.setMetricsEnabled(true) for every
>>> region you
>>> > want to collect the metrics for.
>>> > 2. Use the DataRegionMetricsMXBean.enableMetrics() method exposed by a
>>> > special JMX bean.
>>> >
>>> > More information regarding cache metrics available here:
>>> > https://apacheignite.readme.io/docs/cache-metrics
>>> >
>>> > Regards,
>>> > Igor
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>
>>> This message and any attachments (the "message") is
>>> intended solely for the intended addressees and is confidential.
>>> If you receive this message in error,or are not the intended
>>> recipient(s),
>>> please delete it and any copies from your systems and immediately notify
>>> the sender. Any unauthorized view, use that does not comply with its
>>> purpose,
>>> dissemination or disclosure, either whole or partial, is prohibited.
>>> Since the internet
>>> cannot guarantee the integrity of this message which may not be
>>> reliable, BNP PARIBAS
>>> (and its subsidiaries) shall not be liable for the message if modified,
>>> changed or falsified.
>>> Do not print this message unless it is necessary, consider the
>>> environment.
>>>
>>>
>>> ----------------------------------------------------------------------------------------------------------------------------------
>>>
>>> Ce message et toutes les pieces jointes (ci-apres le "message")
>>> sont etablis a l'intention exclusive de ses destinataires et sont
>>> confidentiels.
>>> Si vous recevez ce message par erreur ou s'il ne vous est pas destine,
>>> merci de le detruire ainsi que toute copie de votre systeme et d'en
>>> avertir
>>> immediatement l'expediteur. Toute lecture non autorisee, toute
>>> utilisation de
>>> ce message qui n'est pas conforme a sa destination, toute diffusion ou
>>> toute
>>> publication, totale ou partielle, est interdite. L'Internet ne
>>> permettant pas d'assurer
>>> l'integrite de ce message electronique susceptible d'alteration, BNP
>>> Paribas
>>> (et ses filiales) decline(nt) toute responsabilite au titre de ce
>>> message dans l'hypothese
>>> ou il aurait ete modifie, deforme ou falsifie.
>>> N'imprimez ce message que si necessaire, pensez a l'environnement.
>>>
>>
Re: How to know memory used by a cache or a set
Posted by Denis Magda <dm...@apache.org>.
Looks like a mess.
Alex Goncharuk, Nikolay Izhikov, considering the latest changes and new
metrics & tracing framework, how would they one get cache/table memory size
via a simple metric? There should be a way to make it workable without
hacks like cachegroups, etc.
-
Denis
On Wed, Jun 19, 2019 at 3:29 AM Alex Plehanov <pl...@gmail.com>
wrote:
> Denis,
>
> Documentation for memory usage calculation covers another case (memory
> usage by the node).
> There is no ability (AFAIK) in released Ignite versions to calculate
> memory used by a cache or cache group when persistence is disabled.
> Dedicated data region can be used for some of the caches in some cases and
> metrics can be collected for this data region, but when the cache is
> destroyed (or data is deleted) memory is not deallocated, it's going to
> reuse list.
> There is a new metric implemented DataRegionMetrics#getTotalUsedPages
> (count of allocated pages minus count of pages in the reuse list) which
> will partially help to solve Yann's problem, but this metric will be
> available only in the next Ignite release.
> Also, as a temporary workaround, some internal API can be used to get a
> count of pages in the reuse list and calculate total used pages by data
> region manually.
>
> ср, 19 июн. 2019 г. в 07:59, Denis Magda <dm...@gridgain.com>:
>
>> + dev list
>>
>> Ignite developers,
>>
>> Seems that the present solution for memory calculation doesn't work (check
>> the thread):
>>
>> https://apacheignite.readme.io/v2.5/docs/memory-metrics#section-memory-usage-calculation
>> <https://apacheignite.readme.io/docs/cache-metrics>
>>
>> Was it really broken?
>>
>> --
>> Denis Magda
>>
>>
>> On Thu, Jun 13, 2019 at 9:45 AM yann Blazart <ya...@gmail.com>
>> wrote:
>>
>> > Ok, but I can't create dynamically a data region ? Because each time I
>> > receive a new file to process, I create a cachegroup to handle it, then
>> I
>> > clean everything.
>> >
>> > Le jeu. 13 juin 2019 à 13:28, Alex Plehanov <pl...@gmail.com> a
>> > écrit :
>> >
>> >> Hello,
>> >>
>> >> It's a known issue [1]. Now you can get cache group size via JMX only
>> if
>> >> persistence is used.
>> >> If persistence is not used you can get allocated size only for data
>> >> region (but you can have more then one data region and assign cache
>> groups
>> >> to data regions any way you want)
>> >>
>> >> [1] : https://issues.apache.org/jira/browse/IGNITE-8517
>> >>
>> >> ср, 12 июн. 2019 г. в 16:00, yann.blazart@externe.bnpparibas.com <
>> >> yann.blazart@externe.bnpparibas.com>:
>> >>
>> >>> Hello, I'm back.
>> >>>
>> >>> Well, I need to get memory used by each execution of my process, so I
>> >>> put all involved caches into the same cacheGroup.
>> >>>
>> >>> If I use the CacheGroupMetricsBean, the size gave to me is 0 !
>> >>> If I enable persistence on DataRegion, I get size, but I don't want to
>> >>> use the persistence enabled.
>> >>>
>> >>> Is it a bug ?
>> >>>
>> >>> regards.
>> >>>
>> >>> > Le 27 mai 2019 à 11:09, ibelyakov <ig...@gmail.com> a
>> écrit
>> >>> :
>> >>> >
>> >>> > Hi,
>> >>> >
>> >>> > Did you turn on cache metrics for your data region?
>> >>> >
>> >>> > To turn the metrics on, use one of the following approaches:
>> >>> > 1. Set DataRegionConfiguration.setMetricsEnabled(true) for every
>> >>> region you
>> >>> > want to collect the metrics for.
>> >>> > 2. Use the DataRegionMetricsMXBean.enableMetrics() method exposed
>> by a
>> >>> > special JMX bean.
>> >>> >
>> >>> > More information regarding cache metrics available here:
>> >>> > https://apacheignite.readme.io/docs/cache-metrics
>> >>> >
>> >>> > Regards,
>> >>> > Igor
>> >>> >
>> >>> >
>> >>> >
>> >>> >
>> >>> > --
>> >>> > Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>> >>>
>> >>> This message and any attachments (the "message") is
>> >>> intended solely for the intended addressees and is confidential.
>> >>> If you receive this message in error,or are not the intended
>> >>> recipient(s),
>> >>> please delete it and any copies from your systems and immediately
>> notify
>> >>> the sender. Any unauthorized view, use that does not comply with its
>> >>> purpose,
>> >>> dissemination or disclosure, either whole or partial, is prohibited.
>> >>> Since the internet
>> >>> cannot guarantee the integrity of this message which may not be
>> >>> reliable, BNP PARIBAS
>> >>> (and its subsidiaries) shall not be liable for the message if
>> modified,
>> >>> changed or falsified.
>> >>> Do not print this message unless it is necessary, consider the
>> >>> environment.
>> >>>
>> >>>
>> >>>
>> ----------------------------------------------------------------------------------------------------------------------------------
>> >>>
>> >>> Ce message et toutes les pieces jointes (ci-apres le "message")
>> >>> sont etablis a l'intention exclusive de ses destinataires et sont
>> >>> confidentiels.
>> >>> Si vous recevez ce message par erreur ou s'il ne vous est pas destine,
>> >>> merci de le detruire ainsi que toute copie de votre systeme et d'en
>> >>> avertir
>> >>> immediatement l'expediteur. Toute lecture non autorisee, toute
>> >>> utilisation de
>> >>> ce message qui n'est pas conforme a sa destination, toute diffusion ou
>> >>> toute
>> >>> publication, totale ou partielle, est interdite. L'Internet ne
>> >>> permettant pas d'assurer
>> >>> l'integrite de ce message electronique susceptible d'alteration, BNP
>> >>> Paribas
>> >>> (et ses filiales) decline(nt) toute responsabilite au titre de ce
>> >>> message dans l'hypothese
>> >>> ou il aurait ete modifie, deforme ou falsifie.
>> >>> N'imprimez ce message que si necessaire, pensez a l'environnement.
>> >>>
>> >>
>>
>
Re: How to know memory used by a cache or a set
Posted by Denis Magda <dm...@apache.org>.
Looks like a mess.
Alex Goncharuk, Nikolay Izhikov, considering the latest changes and new
metrics & tracing framework, how would they one get cache/table memory size
via a simple metric? There should be a way to make it workable without
hacks like cachegroups, etc.
-
Denis
On Wed, Jun 19, 2019 at 3:29 AM Alex Plehanov <pl...@gmail.com>
wrote:
> Denis,
>
> Documentation for memory usage calculation covers another case (memory
> usage by the node).
> There is no ability (AFAIK) in released Ignite versions to calculate
> memory used by a cache or cache group when persistence is disabled.
> Dedicated data region can be used for some of the caches in some cases and
> metrics can be collected for this data region, but when the cache is
> destroyed (or data is deleted) memory is not deallocated, it's going to
> reuse list.
> There is a new metric implemented DataRegionMetrics#getTotalUsedPages
> (count of allocated pages minus count of pages in the reuse list) which
> will partially help to solve Yann's problem, but this metric will be
> available only in the next Ignite release.
> Also, as a temporary workaround, some internal API can be used to get a
> count of pages in the reuse list and calculate total used pages by data
> region manually.
>
> ср, 19 июн. 2019 г. в 07:59, Denis Magda <dm...@gridgain.com>:
>
>> + dev list
>>
>> Ignite developers,
>>
>> Seems that the present solution for memory calculation doesn't work (check
>> the thread):
>>
>> https://apacheignite.readme.io/v2.5/docs/memory-metrics#section-memory-usage-calculation
>> <https://apacheignite.readme.io/docs/cache-metrics>
>>
>> Was it really broken?
>>
>> --
>> Denis Magda
>>
>>
>> On Thu, Jun 13, 2019 at 9:45 AM yann Blazart <ya...@gmail.com>
>> wrote:
>>
>> > Ok, but I can't create dynamically a data region ? Because each time I
>> > receive a new file to process, I create a cachegroup to handle it, then
>> I
>> > clean everything.
>> >
>> > Le jeu. 13 juin 2019 à 13:28, Alex Plehanov <pl...@gmail.com> a
>> > écrit :
>> >
>> >> Hello,
>> >>
>> >> It's a known issue [1]. Now you can get cache group size via JMX only
>> if
>> >> persistence is used.
>> >> If persistence is not used you can get allocated size only for data
>> >> region (but you can have more then one data region and assign cache
>> groups
>> >> to data regions any way you want)
>> >>
>> >> [1] : https://issues.apache.org/jira/browse/IGNITE-8517
>> >>
>> >> ср, 12 июн. 2019 г. в 16:00, yann.blazart@externe.bnpparibas.com <
>> >> yann.blazart@externe.bnpparibas.com>:
>> >>
>> >>> Hello, I'm back.
>> >>>
>> >>> Well, I need to get memory used by each execution of my process, so I
>> >>> put all involved caches into the same cacheGroup.
>> >>>
>> >>> If I use the CacheGroupMetricsBean, the size gave to me is 0 !
>> >>> If I enable persistence on DataRegion, I get size, but I don't want to
>> >>> use the persistence enabled.
>> >>>
>> >>> Is it a bug ?
>> >>>
>> >>> regards.
>> >>>
>> >>> > Le 27 mai 2019 à 11:09, ibelyakov <ig...@gmail.com> a
>> écrit
>> >>> :
>> >>> >
>> >>> > Hi,
>> >>> >
>> >>> > Did you turn on cache metrics for your data region?
>> >>> >
>> >>> > To turn the metrics on, use one of the following approaches:
>> >>> > 1. Set DataRegionConfiguration.setMetricsEnabled(true) for every
>> >>> region you
>> >>> > want to collect the metrics for.
>> >>> > 2. Use the DataRegionMetricsMXBean.enableMetrics() method exposed
>> by a
>> >>> > special JMX bean.
>> >>> >
>> >>> > More information regarding cache metrics available here:
>> >>> > https://apacheignite.readme.io/docs/cache-metrics
>> >>> >
>> >>> > Regards,
>> >>> > Igor
>> >>> >
>> >>> >
>> >>> >
>> >>> >
>> >>> > --
>> >>> > Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>> >>>
>> >>> This message and any attachments (the "message") is
>> >>> intended solely for the intended addressees and is confidential.
>> >>> If you receive this message in error,or are not the intended
>> >>> recipient(s),
>> >>> please delete it and any copies from your systems and immediately
>> notify
>> >>> the sender. Any unauthorized view, use that does not comply with its
>> >>> purpose,
>> >>> dissemination or disclosure, either whole or partial, is prohibited.
>> >>> Since the internet
>> >>> cannot guarantee the integrity of this message which may not be
>> >>> reliable, BNP PARIBAS
>> >>> (and its subsidiaries) shall not be liable for the message if
>> modified,
>> >>> changed or falsified.
>> >>> Do not print this message unless it is necessary, consider the
>> >>> environment.
>> >>>
>> >>>
>> >>>
>> ----------------------------------------------------------------------------------------------------------------------------------
>> >>>
>> >>> Ce message et toutes les pieces jointes (ci-apres le "message")
>> >>> sont etablis a l'intention exclusive de ses destinataires et sont
>> >>> confidentiels.
>> >>> Si vous recevez ce message par erreur ou s'il ne vous est pas destine,
>> >>> merci de le detruire ainsi que toute copie de votre systeme et d'en
>> >>> avertir
>> >>> immediatement l'expediteur. Toute lecture non autorisee, toute
>> >>> utilisation de
>> >>> ce message qui n'est pas conforme a sa destination, toute diffusion ou
>> >>> toute
>> >>> publication, totale ou partielle, est interdite. L'Internet ne
>> >>> permettant pas d'assurer
>> >>> l'integrite de ce message electronique susceptible d'alteration, BNP
>> >>> Paribas
>> >>> (et ses filiales) decline(nt) toute responsabilite au titre de ce
>> >>> message dans l'hypothese
>> >>> ou il aurait ete modifie, deforme ou falsifie.
>> >>> N'imprimez ce message que si necessaire, pensez a l'environnement.
>> >>>
>> >>
>>
>
Re: How to know memory used by a cache or a set
Posted by Alex Plehanov <pl...@gmail.com>.
Denis,
Documentation for memory usage calculation covers another case (memory
usage by the node).
There is no ability (AFAIK) in released Ignite versions to calculate memory
used by a cache or cache group when persistence is disabled.
Dedicated data region can be used for some of the caches in some cases and
metrics can be collected for this data region, but when the cache is
destroyed (or data is deleted) memory is not deallocated, it's going to
reuse list.
There is a new metric implemented DataRegionMetrics#getTotalUsedPages
(count of allocated pages minus count of pages in the reuse list) which
will partially help to solve Yann's problem, but this metric will be
available only in the next Ignite release.
Also, as a temporary workaround, some internal API can be used to get a
count of pages in the reuse list and calculate total used pages by data
region manually.
ср, 19 июн. 2019 г. в 07:59, Denis Magda <dm...@gridgain.com>:
> + dev list
>
> Ignite developers,
>
> Seems that the present solution for memory calculation doesn't work (check
> the thread):
>
> https://apacheignite.readme.io/v2.5/docs/memory-metrics#section-memory-usage-calculation
> <https://apacheignite.readme.io/docs/cache-metrics>
>
> Was it really broken?
>
> --
> Denis Magda
>
>
> On Thu, Jun 13, 2019 at 9:45 AM yann Blazart <ya...@gmail.com>
> wrote:
>
> > Ok, but I can't create dynamically a data region ? Because each time I
> > receive a new file to process, I create a cachegroup to handle it, then I
> > clean everything.
> >
> > Le jeu. 13 juin 2019 à 13:28, Alex Plehanov <pl...@gmail.com> a
> > écrit :
> >
> >> Hello,
> >>
> >> It's a known issue [1]. Now you can get cache group size via JMX only if
> >> persistence is used.
> >> If persistence is not used you can get allocated size only for data
> >> region (but you can have more then one data region and assign cache
> groups
> >> to data regions any way you want)
> >>
> >> [1] : https://issues.apache.org/jira/browse/IGNITE-8517
> >>
> >> ср, 12 июн. 2019 г. в 16:00, yann.blazart@externe.bnpparibas.com <
> >> yann.blazart@externe.bnpparibas.com>:
> >>
> >>> Hello, I'm back.
> >>>
> >>> Well, I need to get memory used by each execution of my process, so I
> >>> put all involved caches into the same cacheGroup.
> >>>
> >>> If I use the CacheGroupMetricsBean, the size gave to me is 0 !
> >>> If I enable persistence on DataRegion, I get size, but I don't want to
> >>> use the persistence enabled.
> >>>
> >>> Is it a bug ?
> >>>
> >>> regards.
> >>>
> >>> > Le 27 mai 2019 à 11:09, ibelyakov <ig...@gmail.com> a
> écrit
> >>> :
> >>> >
> >>> > Hi,
> >>> >
> >>> > Did you turn on cache metrics for your data region?
> >>> >
> >>> > To turn the metrics on, use one of the following approaches:
> >>> > 1. Set DataRegionConfiguration.setMetricsEnabled(true) for every
> >>> region you
> >>> > want to collect the metrics for.
> >>> > 2. Use the DataRegionMetricsMXBean.enableMetrics() method exposed by
> a
> >>> > special JMX bean.
> >>> >
> >>> > More information regarding cache metrics available here:
> >>> > https://apacheignite.readme.io/docs/cache-metrics
> >>> >
> >>> > Regards,
> >>> > Igor
> >>> >
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > Sent from: http://apache-ignite-users.70518.x6.nabble.com/
> >>>
> >>> This message and any attachments (the "message") is
> >>> intended solely for the intended addressees and is confidential.
> >>> If you receive this message in error,or are not the intended
> >>> recipient(s),
> >>> please delete it and any copies from your systems and immediately
> notify
> >>> the sender. Any unauthorized view, use that does not comply with its
> >>> purpose,
> >>> dissemination or disclosure, either whole or partial, is prohibited.
> >>> Since the internet
> >>> cannot guarantee the integrity of this message which may not be
> >>> reliable, BNP PARIBAS
> >>> (and its subsidiaries) shall not be liable for the message if modified,
> >>> changed or falsified.
> >>> Do not print this message unless it is necessary, consider the
> >>> environment.
> >>>
> >>>
> >>>
> ----------------------------------------------------------------------------------------------------------------------------------
> >>>
> >>> Ce message et toutes les pieces jointes (ci-apres le "message")
> >>> sont etablis a l'intention exclusive de ses destinataires et sont
> >>> confidentiels.
> >>> Si vous recevez ce message par erreur ou s'il ne vous est pas destine,
> >>> merci de le detruire ainsi que toute copie de votre systeme et d'en
> >>> avertir
> >>> immediatement l'expediteur. Toute lecture non autorisee, toute
> >>> utilisation de
> >>> ce message qui n'est pas conforme a sa destination, toute diffusion ou
> >>> toute
> >>> publication, totale ou partielle, est interdite. L'Internet ne
> >>> permettant pas d'assurer
> >>> l'integrite de ce message electronique susceptible d'alteration, BNP
> >>> Paribas
> >>> (et ses filiales) decline(nt) toute responsabilite au titre de ce
> >>> message dans l'hypothese
> >>> ou il aurait ete modifie, deforme ou falsifie.
> >>> N'imprimez ce message que si necessaire, pensez a l'environnement.
> >>>
> >>
>
Re: How to know memory used by a cache or a set
Posted by Alex Plehanov <pl...@gmail.com>.
Denis,
Documentation for memory usage calculation covers another case (memory
usage by the node).
There is no ability (AFAIK) in released Ignite versions to calculate memory
used by a cache or cache group when persistence is disabled.
Dedicated data region can be used for some of the caches in some cases and
metrics can be collected for this data region, but when the cache is
destroyed (or data is deleted) memory is not deallocated, it's going to
reuse list.
There is a new metric implemented DataRegionMetrics#getTotalUsedPages
(count of allocated pages minus count of pages in the reuse list) which
will partially help to solve Yann's problem, but this metric will be
available only in the next Ignite release.
Also, as a temporary workaround, some internal API can be used to get a
count of pages in the reuse list and calculate total used pages by data
region manually.
ср, 19 июн. 2019 г. в 07:59, Denis Magda <dm...@gridgain.com>:
> + dev list
>
> Ignite developers,
>
> Seems that the present solution for memory calculation doesn't work (check
> the thread):
>
> https://apacheignite.readme.io/v2.5/docs/memory-metrics#section-memory-usage-calculation
> <https://apacheignite.readme.io/docs/cache-metrics>
>
> Was it really broken?
>
> --
> Denis Magda
>
>
> On Thu, Jun 13, 2019 at 9:45 AM yann Blazart <ya...@gmail.com>
> wrote:
>
> > Ok, but I can't create dynamically a data region ? Because each time I
> > receive a new file to process, I create a cachegroup to handle it, then I
> > clean everything.
> >
> > Le jeu. 13 juin 2019 à 13:28, Alex Plehanov <pl...@gmail.com> a
> > écrit :
> >
> >> Hello,
> >>
> >> It's a known issue [1]. Now you can get cache group size via JMX only if
> >> persistence is used.
> >> If persistence is not used you can get allocated size only for data
> >> region (but you can have more then one data region and assign cache
> groups
> >> to data regions any way you want)
> >>
> >> [1] : https://issues.apache.org/jira/browse/IGNITE-8517
> >>
> >> ср, 12 июн. 2019 г. в 16:00, yann.blazart@externe.bnpparibas.com <
> >> yann.blazart@externe.bnpparibas.com>:
> >>
> >>> Hello, I'm back.
> >>>
> >>> Well, I need to get memory used by each execution of my process, so I
> >>> put all involved caches into the same cacheGroup.
> >>>
> >>> If I use the CacheGroupMetricsBean, the size gave to me is 0 !
> >>> If I enable persistence on DataRegion, I get size, but I don't want to
> >>> use the persistence enabled.
> >>>
> >>> Is it a bug ?
> >>>
> >>> regards.
> >>>
> >>> > Le 27 mai 2019 à 11:09, ibelyakov <ig...@gmail.com> a
> écrit
> >>> :
> >>> >
> >>> > Hi,
> >>> >
> >>> > Did you turn on cache metrics for your data region?
> >>> >
> >>> > To turn the metrics on, use one of the following approaches:
> >>> > 1. Set DataRegionConfiguration.setMetricsEnabled(true) for every
> >>> region you
> >>> > want to collect the metrics for.
> >>> > 2. Use the DataRegionMetricsMXBean.enableMetrics() method exposed by
> a
> >>> > special JMX bean.
> >>> >
> >>> > More information regarding cache metrics available here:
> >>> > https://apacheignite.readme.io/docs/cache-metrics
> >>> >
> >>> > Regards,
> >>> > Igor
> >>> >
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > Sent from: http://apache-ignite-users.70518.x6.nabble.com/
> >>>
> >>> This message and any attachments (the "message") is
> >>> intended solely for the intended addressees and is confidential.
> >>> If you receive this message in error,or are not the intended
> >>> recipient(s),
> >>> please delete it and any copies from your systems and immediately
> notify
> >>> the sender. Any unauthorized view, use that does not comply with its
> >>> purpose,
> >>> dissemination or disclosure, either whole or partial, is prohibited.
> >>> Since the internet
> >>> cannot guarantee the integrity of this message which may not be
> >>> reliable, BNP PARIBAS
> >>> (and its subsidiaries) shall not be liable for the message if modified,
> >>> changed or falsified.
> >>> Do not print this message unless it is necessary, consider the
> >>> environment.
> >>>
> >>>
> >>>
> ----------------------------------------------------------------------------------------------------------------------------------
> >>>
> >>> Ce message et toutes les pieces jointes (ci-apres le "message")
> >>> sont etablis a l'intention exclusive de ses destinataires et sont
> >>> confidentiels.
> >>> Si vous recevez ce message par erreur ou s'il ne vous est pas destine,
> >>> merci de le detruire ainsi que toute copie de votre systeme et d'en
> >>> avertir
> >>> immediatement l'expediteur. Toute lecture non autorisee, toute
> >>> utilisation de
> >>> ce message qui n'est pas conforme a sa destination, toute diffusion ou
> >>> toute
> >>> publication, totale ou partielle, est interdite. L'Internet ne
> >>> permettant pas d'assurer
> >>> l'integrite de ce message electronique susceptible d'alteration, BNP
> >>> Paribas
> >>> (et ses filiales) decline(nt) toute responsabilite au titre de ce
> >>> message dans l'hypothese
> >>> ou il aurait ete modifie, deforme ou falsifie.
> >>> N'imprimez ce message que si necessaire, pensez a l'environnement.
> >>>
> >>
>
Re: How to know memory used by a cache or a set
Posted by Denis Magda <dm...@gridgain.com>.
+ dev list
Ignite developers,
Seems that the present solution for memory calculation doesn't work (check
the thread):
https://apacheignite.readme.io/v2.5/docs/memory-metrics#section-memory-usage-calculation
<https://apacheignite.readme.io/docs/cache-metrics>
Was it really broken?
--
Denis Magda
On Thu, Jun 13, 2019 at 9:45 AM yann Blazart <ya...@gmail.com> wrote:
> Ok, but I can't create dynamically a data region ? Because each time I
> receive a new file to process, I create a cachegroup to handle it, then I
> clean everything.
>
> Le jeu. 13 juin 2019 à 13:28, Alex Plehanov <pl...@gmail.com> a
> écrit :
>
>> Hello,
>>
>> It's a known issue [1]. Now you can get cache group size via JMX only if
>> persistence is used.
>> If persistence is not used you can get allocated size only for data
>> region (but you can have more then one data region and assign cache groups
>> to data regions any way you want)
>>
>> [1] : https://issues.apache.org/jira/browse/IGNITE-8517
>>
>> ср, 12 июн. 2019 г. в 16:00, yann.blazart@externe.bnpparibas.com <
>> yann.blazart@externe.bnpparibas.com>:
>>
>>> Hello, I'm back.
>>>
>>> Well, I need to get memory used by each execution of my process, so I
>>> put all involved caches into the same cacheGroup.
>>>
>>> If I use the CacheGroupMetricsBean, the size gave to me is 0 !
>>> If I enable persistence on DataRegion, I get size, but I don't want to
>>> use the persistence enabled.
>>>
>>> Is it a bug ?
>>>
>>> regards.
>>>
>>> > Le 27 mai 2019 à 11:09, ibelyakov <ig...@gmail.com> a écrit
>>> :
>>> >
>>> > Hi,
>>> >
>>> > Did you turn on cache metrics for your data region?
>>> >
>>> > To turn the metrics on, use one of the following approaches:
>>> > 1. Set DataRegionConfiguration.setMetricsEnabled(true) for every
>>> region you
>>> > want to collect the metrics for.
>>> > 2. Use the DataRegionMetricsMXBean.enableMetrics() method exposed by a
>>> > special JMX bean.
>>> >
>>> > More information regarding cache metrics available here:
>>> > https://apacheignite.readme.io/docs/cache-metrics
>>> >
>>> > Regards,
>>> > Igor
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>
>>> This message and any attachments (the "message") is
>>> intended solely for the intended addressees and is confidential.
>>> If you receive this message in error,or are not the intended
>>> recipient(s),
>>> please delete it and any copies from your systems and immediately notify
>>> the sender. Any unauthorized view, use that does not comply with its
>>> purpose,
>>> dissemination or disclosure, either whole or partial, is prohibited.
>>> Since the internet
>>> cannot guarantee the integrity of this message which may not be
>>> reliable, BNP PARIBAS
>>> (and its subsidiaries) shall not be liable for the message if modified,
>>> changed or falsified.
>>> Do not print this message unless it is necessary, consider the
>>> environment.
>>>
>>>
>>> ----------------------------------------------------------------------------------------------------------------------------------
>>>
>>> Ce message et toutes les pieces jointes (ci-apres le "message")
>>> sont etablis a l'intention exclusive de ses destinataires et sont
>>> confidentiels.
>>> Si vous recevez ce message par erreur ou s'il ne vous est pas destine,
>>> merci de le detruire ainsi que toute copie de votre systeme et d'en
>>> avertir
>>> immediatement l'expediteur. Toute lecture non autorisee, toute
>>> utilisation de
>>> ce message qui n'est pas conforme a sa destination, toute diffusion ou
>>> toute
>>> publication, totale ou partielle, est interdite. L'Internet ne
>>> permettant pas d'assurer
>>> l'integrite de ce message electronique susceptible d'alteration, BNP
>>> Paribas
>>> (et ses filiales) decline(nt) toute responsabilite au titre de ce
>>> message dans l'hypothese
>>> ou il aurait ete modifie, deforme ou falsifie.
>>> N'imprimez ce message que si necessaire, pensez a l'environnement.
>>>
>>
Re: How to know memory used by a cache or a set
Posted by Denis Magda <dm...@gridgain.com>.
+ dev list
Ignite developers,
Seems that the present solution for memory calculation doesn't work (check
the thread):
https://apacheignite.readme.io/v2.5/docs/memory-metrics#section-memory-usage-calculation
<https://apacheignite.readme.io/docs/cache-metrics>
Was it really broken?
--
Denis Magda
On Thu, Jun 13, 2019 at 9:45 AM yann Blazart <ya...@gmail.com> wrote:
> Ok, but I can't create dynamically a data region ? Because each time I
> receive a new file to process, I create a cachegroup to handle it, then I
> clean everything.
>
> Le jeu. 13 juin 2019 à 13:28, Alex Plehanov <pl...@gmail.com> a
> écrit :
>
>> Hello,
>>
>> It's a known issue [1]. Now you can get cache group size via JMX only if
>> persistence is used.
>> If persistence is not used you can get allocated size only for data
>> region (but you can have more then one data region and assign cache groups
>> to data regions any way you want)
>>
>> [1] : https://issues.apache.org/jira/browse/IGNITE-8517
>>
>> ср, 12 июн. 2019 г. в 16:00, yann.blazart@externe.bnpparibas.com <
>> yann.blazart@externe.bnpparibas.com>:
>>
>>> Hello, I'm back.
>>>
>>> Well, I need to get memory used by each execution of my process, so I
>>> put all involved caches into the same cacheGroup.
>>>
>>> If I use the CacheGroupMetricsBean, the size gave to me is 0 !
>>> If I enable persistence on DataRegion, I get size, but I don't want to
>>> use the persistence enabled.
>>>
>>> Is it a bug ?
>>>
>>> regards.
>>>
>>> > Le 27 mai 2019 à 11:09, ibelyakov <ig...@gmail.com> a écrit
>>> :
>>> >
>>> > Hi,
>>> >
>>> > Did you turn on cache metrics for your data region?
>>> >
>>> > To turn the metrics on, use one of the following approaches:
>>> > 1. Set DataRegionConfiguration.setMetricsEnabled(true) for every
>>> region you
>>> > want to collect the metrics for.
>>> > 2. Use the DataRegionMetricsMXBean.enableMetrics() method exposed by a
>>> > special JMX bean.
>>> >
>>> > More information regarding cache metrics available here:
>>> > https://apacheignite.readme.io/docs/cache-metrics
>>> >
>>> > Regards,
>>> > Igor
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>
>>> This message and any attachments (the "message") is
>>> intended solely for the intended addressees and is confidential.
>>> If you receive this message in error,or are not the intended
>>> recipient(s),
>>> please delete it and any copies from your systems and immediately notify
>>> the sender. Any unauthorized view, use that does not comply with its
>>> purpose,
>>> dissemination or disclosure, either whole or partial, is prohibited.
>>> Since the internet
>>> cannot guarantee the integrity of this message which may not be
>>> reliable, BNP PARIBAS
>>> (and its subsidiaries) shall not be liable for the message if modified,
>>> changed or falsified.
>>> Do not print this message unless it is necessary, consider the
>>> environment.
>>>
>>>
>>> ----------------------------------------------------------------------------------------------------------------------------------
>>>
>>> Ce message et toutes les pieces jointes (ci-apres le "message")
>>> sont etablis a l'intention exclusive de ses destinataires et sont
>>> confidentiels.
>>> Si vous recevez ce message par erreur ou s'il ne vous est pas destine,
>>> merci de le detruire ainsi que toute copie de votre systeme et d'en
>>> avertir
>>> immediatement l'expediteur. Toute lecture non autorisee, toute
>>> utilisation de
>>> ce message qui n'est pas conforme a sa destination, toute diffusion ou
>>> toute
>>> publication, totale ou partielle, est interdite. L'Internet ne
>>> permettant pas d'assurer
>>> l'integrite de ce message electronique susceptible d'alteration, BNP
>>> Paribas
>>> (et ses filiales) decline(nt) toute responsabilite au titre de ce
>>> message dans l'hypothese
>>> ou il aurait ete modifie, deforme ou falsifie.
>>> N'imprimez ce message que si necessaire, pensez a l'environnement.
>>>
>>
Re: How to know memory used by a cache or a set
Posted by yann Blazart <ya...@gmail.com>.
Ok, but I can't create dynamically a data region ? Because each time I
receive a new file to process, I create a cachegroup to handle it, then I
clean everything.
Le jeu. 13 juin 2019 à 13:28, Alex Plehanov <pl...@gmail.com> a
écrit :
> Hello,
>
> It's a known issue [1]. Now you can get cache group size via JMX only if
> persistence is used.
> If persistence is not used you can get allocated size only for data region
> (but you can have more then one data region and assign cache groups to data
> regions any way you want)
>
> [1] : https://issues.apache.org/jira/browse/IGNITE-8517
>
> ср, 12 июн. 2019 г. в 16:00, yann.blazart@externe.bnpparibas.com <
> yann.blazart@externe.bnpparibas.com>:
>
>> Hello, I'm back.
>>
>> Well, I need to get memory used by each execution of my process, so I put
>> all involved caches into the same cacheGroup.
>>
>> If I use the CacheGroupMetricsBean, the size gave to me is 0 !
>> If I enable persistence on DataRegion, I get size, but I don't want to
>> use the persistence enabled.
>>
>> Is it a bug ?
>>
>> regards.
>>
>> > Le 27 mai 2019 à 11:09, ibelyakov <ig...@gmail.com> a écrit :
>> >
>> > Hi,
>> >
>> > Did you turn on cache metrics for your data region?
>> >
>> > To turn the metrics on, use one of the following approaches:
>> > 1. Set DataRegionConfiguration.setMetricsEnabled(true) for every region
>> you
>> > want to collect the metrics for.
>> > 2. Use the DataRegionMetricsMXBean.enableMetrics() method exposed by a
>> > special JMX bean.
>> >
>> > More information regarding cache metrics available here:
>> > https://apacheignite.readme.io/docs/cache-metrics
>> >
>> > Regards,
>> > Igor
>> >
>> >
>> >
>> >
>> > --
>> > Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>> This message and any attachments (the "message") is
>> intended solely for the intended addressees and is confidential.
>> If you receive this message in error,or are not the intended
>> recipient(s),
>> please delete it and any copies from your systems and immediately notify
>> the sender. Any unauthorized view, use that does not comply with its
>> purpose,
>> dissemination or disclosure, either whole or partial, is prohibited.
>> Since the internet
>> cannot guarantee the integrity of this message which may not be reliable,
>> BNP PARIBAS
>> (and its subsidiaries) shall not be liable for the message if modified,
>> changed or falsified.
>> Do not print this message unless it is necessary, consider the
>> environment.
>>
>>
>> ----------------------------------------------------------------------------------------------------------------------------------
>>
>> Ce message et toutes les pieces jointes (ci-apres le "message")
>> sont etablis a l'intention exclusive de ses destinataires et sont
>> confidentiels.
>> Si vous recevez ce message par erreur ou s'il ne vous est pas destine,
>> merci de le detruire ainsi que toute copie de votre systeme et d'en
>> avertir
>> immediatement l'expediteur. Toute lecture non autorisee, toute
>> utilisation de
>> ce message qui n'est pas conforme a sa destination, toute diffusion ou
>> toute
>> publication, totale ou partielle, est interdite. L'Internet ne permettant
>> pas d'assurer
>> l'integrite de ce message electronique susceptible d'alteration, BNP
>> Paribas
>> (et ses filiales) decline(nt) toute responsabilite au titre de ce message
>> dans l'hypothese
>> ou il aurait ete modifie, deforme ou falsifie.
>> N'imprimez ce message que si necessaire, pensez a l'environnement.
>>
>
Re: How to know memory used by a cache or a set
Posted by Alex Plehanov <pl...@gmail.com>.
Hello,
It's a known issue [1]. Now you can get cache group size via JMX only if
persistence is used.
If persistence is not used you can get allocated size only for data region
(but you can have more then one data region and assign cache groups to data
regions any way you want)
[1] : https://issues.apache.org/jira/browse/IGNITE-8517
ср, 12 июн. 2019 г. в 16:00, yann.blazart@externe.bnpparibas.com <
yann.blazart@externe.bnpparibas.com>:
> Hello, I'm back.
>
> Well, I need to get memory used by each execution of my process, so I put
> all involved caches into the same cacheGroup.
>
> If I use the CacheGroupMetricsBean, the size gave to me is 0 !
> If I enable persistence on DataRegion, I get size, but I don't want to use
> the persistence enabled.
>
> Is it a bug ?
>
> regards.
>
> > Le 27 mai 2019 à 11:09, ibelyakov <ig...@gmail.com> a écrit :
> >
> > Hi,
> >
> > Did you turn on cache metrics for your data region?
> >
> > To turn the metrics on, use one of the following approaches:
> > 1. Set DataRegionConfiguration.setMetricsEnabled(true) for every region
> you
> > want to collect the metrics for.
> > 2. Use the DataRegionMetricsMXBean.enableMetrics() method exposed by a
> > special JMX bean.
> >
> > More information regarding cache metrics available here:
> > https://apacheignite.readme.io/docs/cache-metrics
> >
> > Regards,
> > Igor
> >
> >
> >
> >
> > --
> > Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
> This message and any attachments (the "message") is
> intended solely for the intended addressees and is confidential.
> If you receive this message in error,or are not the intended recipient(s),
> please delete it and any copies from your systems and immediately notify
> the sender. Any unauthorized view, use that does not comply with its
> purpose,
> dissemination or disclosure, either whole or partial, is prohibited. Since
> the internet
> cannot guarantee the integrity of this message which may not be reliable,
> BNP PARIBAS
> (and its subsidiaries) shall not be liable for the message if modified,
> changed or falsified.
> Do not print this message unless it is necessary, consider the environment.
>
>
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Ce message et toutes les pieces jointes (ci-apres le "message")
> sont etablis a l'intention exclusive de ses destinataires et sont
> confidentiels.
> Si vous recevez ce message par erreur ou s'il ne vous est pas destine,
> merci de le detruire ainsi que toute copie de votre systeme et d'en avertir
> immediatement l'expediteur. Toute lecture non autorisee, toute utilisation
> de
> ce message qui n'est pas conforme a sa destination, toute diffusion ou
> toute
> publication, totale ou partielle, est interdite. L'Internet ne permettant
> pas d'assurer
> l'integrite de ce message electronique susceptible d'alteration, BNP
> Paribas
> (et ses filiales) decline(nt) toute responsabilite au titre de ce message
> dans l'hypothese
> ou il aurait ete modifie, deforme ou falsifie.
> N'imprimez ce message que si necessaire, pensez a l'environnement.
>
Re: How to know memory used by a cache or a set
Posted by "yann.blazart@externe.bnpparibas.com" <ya...@externe.bnpparibas.com>.
Hello, I'm back.
Well, I need to get memory used by each execution of my process, so I put all involved caches into the same cacheGroup.
If I use the CacheGroupMetricsBean, the size gave to me is 0 !
If I enable persistence on DataRegion, I get size, but I don't want to use the persistence enabled.
Is it a bug ?
regards.
> Le 27 mai 2019 à 11:09, ibelyakov <ig...@gmail.com> a écrit :
>
> Hi,
>
> Did you turn on cache metrics for your data region?
>
> To turn the metrics on, use one of the following approaches:
> 1. Set DataRegionConfiguration.setMetricsEnabled(true) for every region you
> want to collect the metrics for.
> 2. Use the DataRegionMetricsMXBean.enableMetrics() method exposed by a
> special JMX bean.
>
> More information regarding cache metrics available here:
> https://apacheignite.readme.io/docs/cache-metrics
>
> Regards,
> Igor
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
This message and any attachments (the "message") is
intended solely for the intended addressees and is confidential.
If you receive this message in error,or are not the intended recipient(s),
please delete it and any copies from your systems and immediately notify
the sender. Any unauthorized view, use that does not comply with its purpose,
dissemination or disclosure, either whole or partial, is prohibited. Since the internet
cannot guarantee the integrity of this message which may not be reliable, BNP PARIBAS
(and its subsidiaries) shall not be liable for the message if modified, changed or falsified.
Do not print this message unless it is necessary, consider the environment.
----------------------------------------------------------------------------------------------------------------------------------
Ce message et toutes les pieces jointes (ci-apres le "message")
sont etablis a l'intention exclusive de ses destinataires et sont confidentiels.
Si vous recevez ce message par erreur ou s'il ne vous est pas destine,
merci de le detruire ainsi que toute copie de votre systeme et d'en avertir
immediatement l'expediteur. Toute lecture non autorisee, toute utilisation de
ce message qui n'est pas conforme a sa destination, toute diffusion ou toute
publication, totale ou partielle, est interdite. L'Internet ne permettant pas d'assurer
l'integrite de ce message electronique susceptible d'alteration, BNP Paribas
(et ses filiales) decline(nt) toute responsabilite au titre de ce message dans l'hypothese
ou il aurait ete modifie, deforme ou falsifie.
N'imprimez ce message que si necessaire, pensez a l'environnement.
Re: How to know memory used by a cache or a set
Posted by yann Blazart <ya...@gmail.com>.
Hello. I'm almost sure I already did that. I will check tomorrow.
Thks
Le lun. 27 mai 2019 à 11:09, ibelyakov <ig...@gmail.com> a écrit :
> Hi,
>
> Did you turn on cache metrics for your data region?
>
> To turn the metrics on, use one of the following approaches:
> 1. Set DataRegionConfiguration.setMetricsEnabled(true) for every region you
> want to collect the metrics for.
> 2. Use the DataRegionMetricsMXBean.enableMetrics() method exposed by a
> special JMX bean.
>
> More information regarding cache metrics available here:
> https://apacheignite.readme.io/docs/cache-metrics
>
> Regards,
> Igor
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: How to know memory used by a cache or a set
Posted by ibelyakov <ig...@gmail.com>.
Hi,
Did you turn on cache metrics for your data region?
To turn the metrics on, use one of the following approaches:
1. Set DataRegionConfiguration.setMetricsEnabled(true) for every region you
want to collect the metrics for.
2. Use the DataRegionMetricsMXBean.enableMetrics() method exposed by a
special JMX bean.
More information regarding cache metrics available here:
https://apacheignite.readme.io/docs/cache-metrics
Regards,
Igor
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/