Fwd: Issue 910 in asterixdb: Index creation/use can cause resource leakage

[Michael Carey <mj...@ics.uci.edu>]

Definitely a high priority bug - we should try to fix this pre-release!


-------- Forwarded Message --------
Subject: 	Issue 910 in asterixdb: Index creation/use can cause resource 
leakage
Date: 	Wed, 22 Jul 2015 03:13:59 +0000
From: 	asterixdb@googlecode.com
Reply-To: 	asterixdb-dev@googlegroups.com
To: 	asterixdb-dev@googlegroups.com



Status: Accepted
Owner: ima...@uci.edu
CC: buyingyi@gmail.com
Labels: Type-Defect Priority-High

New issue 910 by ima...@uci.edu: Index creation/use can cause resource
leakage
https://code.google.com/p/asterixdb/issues/detail?id=910

In IndexDataflowHelper, calling create() will make an index instance, and
create it. This inserts an entry into the IndexLifecycleManager's resource
map. Entries in this map are only freed once unregister() is called on an
index. However this only happens when an index is destroyed, not when it is
closed. There is no other way in which references to indexes are garbage
collected (even though there is a reference counter in the IndexInfo).
Therefore creating or using an index uses resources in this map even if
they are not active, and were created during the lifetime of the NC- and
there is no way to free that memory.

-- 
You received this message because this project is configured to send all
issue notifications to this address.
You may adjust your notification preferences at:
https://code.google.com/hosting/settings

-- 
You received this message because you are subscribed to the Google Groups "asterixdb-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to asterixdb-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Issue 910 in asterixdb: Index creation/use can cause resource leakage

[Mike Carey <dt...@gmail.com>]

Ah.  The "leak" I am thinking of is more an unreleased use of memory - 
we should make sure that we activate AND deactivate indexes eventually 
and don't just have more and more in-memory state over time.  We should 
probably do a "lifecycle review" of how we manage (i) metadata cache 
entries, (ii) index metadata, and (iii) in-memory index components - 
anything that has a memory footprint at runtime - and make sure we know 
when each first hits memory and then UN-hits memory at some reasonable 
time.  We should also make sure that not all indexes have in-memory 
components; if we are only reading them we shouldn't need those.  
Perhaps we could do this in a "crowd-sourced" Google doc that identifies 
the key lifecycle events (create index, access dataset for first time, 
access index of dataset for first time, hit limit on number of active 
datasets, ....).

On 7/22/15 12:58 AM, Ian Maxon wrote:
> Index open/close is probably fine. I haven't traced it in a debugger
> or anything, but I believe it would just look up the index in the map
> rather than creating a new object. The only way to create a leak is to
> create many unique indexes within an NC's lifetime.
>
> On Wed, Jul 22, 2015 at 12:39 AM, Mike Carey <dt...@gmail.com> wrote:
>> But my guess is that there's similar unfortunate behavior related to index
>> open and close...?
>> (Which would happen way more regularly?)  Agreed w.r.t. priority if we are
>> in "prototype" mode,
>> but if anyone wanted to go "live" and run for months at a time, this would
>> be a leak we'd feel,
>> potentially.  We need to devise and do some testing around index/memory
>> management and
>> the lifecycle there.  :-)
>>
>> On 7/22/15 12:25 AM, abdullah alamoudi wrote:
>>> It should be fixed but I don't think that it is a release blocker since
>>> this rarely happen and the amount of memory per index instance object is
>>> almost nothing.
>>>
>>> just my 2 cents.
>>>
>>> On Wed, Jul 22, 2015 at 10:10 AM, Michael Carey <mj...@ics.uci.edu>
>>> wrote:
>>>
>>>> Definitely a high priority bug - we should try to fix this pre-release!
>>>>
>>>>
>>>> -------- Forwarded Message --------
>>>> Subject:        Issue 910 in asterixdb: Index creation/use can cause
>>>> resource leakage
>>>> Date:   Wed, 22 Jul 2015 03:13:59 +0000
>>>> From:   asterixdb@googlecode.com
>>>> Reply-To:       asterixdb-dev@googlegroups.com
>>>> To:     asterixdb-dev@googlegroups.com
>>>>
>>>>
>>>>
>>>> Status: Accepted
>>>> Owner: ima...@uci.edu
>>>> CC: buyingyi@gmail.com
>>>> Labels: Type-Defect Priority-High
>>>>
>>>> New issue 910 by ima...@uci.edu: Index creation/use can cause resource
>>>> leakage
>>>> https://code.google.com/p/asterixdb/issues/detail?id=910
>>>>
>>>> In IndexDataflowHelper, calling create() will make an index instance, and
>>>> create it. This inserts an entry into the IndexLifecycleManager's
>>>> resource
>>>> map. Entries in this map are only freed once unregister() is called on an
>>>> index. However this only happens when an index is destroyed, not when it
>>>> is
>>>> closed. There is no other way in which references to indexes are garbage
>>>> collected (even though there is a reference counter in the IndexInfo).
>>>> Therefore creating or using an index uses resources in this map even if
>>>> they are not active, and were created during the lifetime of the NC- and
>>>> there is no way to free that memory.
>>>>
>>>> --
>>>> You received this message because this project is configured to send all
>>>> issue notifications to this address.
>>>> You may adjust your notification preferences at:
>>>> https://code.google.com/hosting/settings
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google Groups
>>>> "asterixdb-dev" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send an
>>>> email to asterixdb-dev+unsubscribe@googlegroups.com.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>>
>>>>
>>>>

Re: Issue 910 in asterixdb: Index creation/use can cause resource leakage

[Ian Maxon <im...@uci.edu>]

Index open/close is probably fine. I haven't traced it in a debugger
or anything, but I believe it would just look up the index in the map
rather than creating a new object. The only way to create a leak is to
create many unique indexes within an NC's lifetime.

On Wed, Jul 22, 2015 at 12:39 AM, Mike Carey <dt...@gmail.com> wrote:
> But my guess is that there's similar unfortunate behavior related to index
> open and close...?
> (Which would happen way more regularly?)  Agreed w.r.t. priority if we are
> in "prototype" mode,
> but if anyone wanted to go "live" and run for months at a time, this would
> be a leak we'd feel,
> potentially.  We need to devise and do some testing around index/memory
> management and
> the lifecycle there.  :-)
>
> On 7/22/15 12:25 AM, abdullah alamoudi wrote:
>>
>> It should be fixed but I don't think that it is a release blocker since
>> this rarely happen and the amount of memory per index instance object is
>> almost nothing.
>>
>> just my 2 cents.
>>
>> On Wed, Jul 22, 2015 at 10:10 AM, Michael Carey <mj...@ics.uci.edu>
>> wrote:
>>
>>> Definitely a high priority bug - we should try to fix this pre-release!
>>>
>>>
>>> -------- Forwarded Message --------
>>> Subject:        Issue 910 in asterixdb: Index creation/use can cause
>>> resource leakage
>>> Date:   Wed, 22 Jul 2015 03:13:59 +0000
>>> From:   asterixdb@googlecode.com
>>> Reply-To:       asterixdb-dev@googlegroups.com
>>> To:     asterixdb-dev@googlegroups.com
>>>
>>>
>>>
>>> Status: Accepted
>>> Owner: ima...@uci.edu
>>> CC: buyingyi@gmail.com
>>> Labels: Type-Defect Priority-High
>>>
>>> New issue 910 by ima...@uci.edu: Index creation/use can cause resource
>>> leakage
>>> https://code.google.com/p/asterixdb/issues/detail?id=910
>>>
>>> In IndexDataflowHelper, calling create() will make an index instance, and
>>> create it. This inserts an entry into the IndexLifecycleManager's
>>> resource
>>> map. Entries in this map are only freed once unregister() is called on an
>>> index. However this only happens when an index is destroyed, not when it
>>> is
>>> closed. There is no other way in which references to indexes are garbage
>>> collected (even though there is a reference counter in the IndexInfo).
>>> Therefore creating or using an index uses resources in this map even if
>>> they are not active, and were created during the lifetime of the NC- and
>>> there is no way to free that memory.
>>>
>>> --
>>> You received this message because this project is configured to send all
>>> issue notifications to this address.
>>> You may adjust your notification preferences at:
>>> https://code.google.com/hosting/settings
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "asterixdb-dev" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to asterixdb-dev+unsubscribe@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>>
>>>
>>
>

Re: Issue 910 in asterixdb: Index creation/use can cause resource leakage

[Mike Carey <dt...@gmail.com>]

But my guess is that there's similar unfortunate behavior related to 
index open and close...?
(Which would happen way more regularly?)  Agreed w.r.t. priority if we 
are in "prototype" mode,
but if anyone wanted to go "live" and run for months at a time, this 
would be a leak we'd feel,
potentially.  We need to devise and do some testing around index/memory 
management and
the lifecycle there.  :-)

On 7/22/15 12:25 AM, abdullah alamoudi wrote:
> It should be fixed but I don't think that it is a release blocker since
> this rarely happen and the amount of memory per index instance object is
> almost nothing.
>
> just my 2 cents.
>
> On Wed, Jul 22, 2015 at 10:10 AM, Michael Carey <mj...@ics.uci.edu> wrote:
>
>> Definitely a high priority bug - we should try to fix this pre-release!
>>
>>
>> -------- Forwarded Message --------
>> Subject:        Issue 910 in asterixdb: Index creation/use can cause
>> resource leakage
>> Date:   Wed, 22 Jul 2015 03:13:59 +0000
>> From:   asterixdb@googlecode.com
>> Reply-To:       asterixdb-dev@googlegroups.com
>> To:     asterixdb-dev@googlegroups.com
>>
>>
>>
>> Status: Accepted
>> Owner: ima...@uci.edu
>> CC: buyingyi@gmail.com
>> Labels: Type-Defect Priority-High
>>
>> New issue 910 by ima...@uci.edu: Index creation/use can cause resource
>> leakage
>> https://code.google.com/p/asterixdb/issues/detail?id=910
>>
>> In IndexDataflowHelper, calling create() will make an index instance, and
>> create it. This inserts an entry into the IndexLifecycleManager's resource
>> map. Entries in this map are only freed once unregister() is called on an
>> index. However this only happens when an index is destroyed, not when it is
>> closed. There is no other way in which references to indexes are garbage
>> collected (even though there is a reference counter in the IndexInfo).
>> Therefore creating or using an index uses resources in this map even if
>> they are not active, and were created during the lifetime of the NC- and
>> there is no way to free that memory.
>>
>> --
>> You received this message because this project is configured to send all
>> issue notifications to this address.
>> You may adjust your notification preferences at:
>> https://code.google.com/hosting/settings
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "asterixdb-dev" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to asterixdb-dev+unsubscribe@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>>
>>
>

Re: Issue 910 in asterixdb: Index creation/use can cause resource leakage

[abdullah alamoudi <ba...@gmail.com>]

It should be fixed but I don't think that it is a release blocker since
this rarely happen and the amount of memory per index instance object is
almost nothing.

just my 2 cents.

On Wed, Jul 22, 2015 at 10:10 AM, Michael Carey <mj...@ics.uci.edu> wrote:

> Definitely a high priority bug - we should try to fix this pre-release!
>
>
> -------- Forwarded Message --------
> Subject:        Issue 910 in asterixdb: Index creation/use can cause
> resource leakage
> Date:   Wed, 22 Jul 2015 03:13:59 +0000
> From:   asterixdb@googlecode.com
> Reply-To:       asterixdb-dev@googlegroups.com
> To:     asterixdb-dev@googlegroups.com
>
>
>
> Status: Accepted
> Owner: ima...@uci.edu
> CC: buyingyi@gmail.com
> Labels: Type-Defect Priority-High
>
> New issue 910 by ima...@uci.edu: Index creation/use can cause resource
> leakage
> https://code.google.com/p/asterixdb/issues/detail?id=910
>
> In IndexDataflowHelper, calling create() will make an index instance, and
> create it. This inserts an entry into the IndexLifecycleManager's resource
> map. Entries in this map are only freed once unregister() is called on an
> index. However this only happens when an index is destroyed, not when it is
> closed. There is no other way in which references to indexes are garbage
> collected (even though there is a reference counter in the IndexInfo).
> Therefore creating or using an index uses resources in this map even if
> they are not active, and were created during the lifetime of the NC- and
> there is no way to free that memory.
>
> --
> You received this message because this project is configured to send all
> issue notifications to this address.
> You may adjust your notification preferences at:
> https://code.google.com/hosting/settings
>
> --
> You received this message because you are subscribed to the Google Groups
> "asterixdb-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to asterixdb-dev+unsubscribe@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
>
>
>


-- 
Amoudi, Abdullah.