You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@trafodion.apache.org by Dave Birdsall <da...@esgyn.com> on 2017/06/08 18:20:51 UTC
NATableDB (NATable cache)
Hi Trafodion developers,
I'm debugging some issues pertaining to NATable descriptors, and need to understand the behavior of the NATable cache (that is, NATableDB). I can figure it out from the code, but it might be faster to pick the collective Trafodion brains.
Some questions about NATableDB:
1. I have the impression that one NATableDB exists per context, and it lives as long as the life of the context. Is this true?
2. It looks like NATable objects might be created on the statement heap or on the context heap. It looks like both may be cached. Is this true?
3. Assuming "yes" on question 2, how do the ones created on the statement heap get removed when the statement heap itself is destroyed?
Thanks,
Dave
RE: NATableDB (NATable cache)
Posted by Selva Govindarajan <se...@esgyn.com>.
It depends upon what context you are referring to. There is only NATable one cache per compiler context (CmpContext is the class). But the execution context (ContextCli is the class) can have two or more CmpContext. - One is user query CmpContext that has the cache of user tables and the other is metadata CmpContext that has the cache of metadata tables. In addition, I have seen a third CmpContext at times. However, when user application issues query against meta data tables, I believe metadata tables involved in the query would be cached in the user CmpContext.
Yes. NATable cache lives as long as the CmpContext. But some of the DDL commands would invalidate the NATable entry corresponding to the table involved in the DDL. There is also a CQD TRAF_RELOAD_NATABLE_CACHE that can invalidate all the tables involved in the subsequent query and refresh those NATable entries.
The CmpContext lives as long as the ContextCli. Most often there is only one ContextCli - a default Context in a process. The default Context lives as long as the process. However, in case of Type 2 jdbc application, there can be as many ContextCli as the number of JDBC connections. These contexts (both execution and the compiler contexts) go away when the connection is closed.
I wish I had simple answer to your question 1.
Selva
-----Original Message-----
From: Dave Birdsall [mailto:dave.birdsall@esgyn.com]
Sent: Thursday, June 8, 2017 11:29 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: NATableDB (NATable cache)
And as soon as I hit "send", I think I found some answers (to questions 2 and 3 anyway).
In the second NATableDB::get function, right after inserting an NATable object in the cache, there is logic like this:
//insert into cache
insert(table); <-- sticks an object in the cache, whether or not it is "cacheable"
//if we are using the cache
//if this NATable object is cacheable
if((useCache_) &&
(corrName.isCacheable()))
{
... do a bunch of stuff such as maintain high water marks ...
}
else{
//this has to be on the context heap since we need
//it after the statement heap has been remove
ExtendedQualName * nonCacheableTableName = new(CmpCommon::contextHeap())
ExtendedQualName(corrName.getExtendedQualNameObj(),
CmpCommon::contextHeap());
//insert into list of names of special tables
nonCacheableTableList_.insert(nonCacheableTableName);
// insert into list of non cacheable table idents. This
// allows the idents to be removed after the statement so
// the context heap doesn't keep growing.
const LIST(CollIndex) & tableIdList = table->getTableIdList();
for(CollIndex i = 0; i < tableIdList.entries(); i++)
{
nonCacheableTableIdents_.insert(tableIdList[i]);
}
}
So, the answer is, everything gets cached. But after the statement heap is destroyed, we come back to NATableDB somehow and remove all the entries that were allocated from the statement heap. For this to work correctly, however, this invariant relation must be maintained: (NATable was allocated on statement heap) if and only if (((useCache_) && (corrName.isCacheable()))
(Maybe it would be simpler and more robust to just check which heap the NATable object was allocated on instead.)
Thanks,
Dave
From: Dave Birdsall
Sent: Thursday, June 8, 2017 11:21 AM
To: dev@trafodion.incubator.apache.org
Subject: NATableDB (NATable cache)
Hi Trafodion developers,
I'm debugging some issues pertaining to NATable descriptors, and need to understand the behavior of the NATable cache (that is, NATableDB). I can figure it out from the code, but it might be faster to pick the collective Trafodion brains.
Some questions about NATableDB:
1. I have the impression that one NATableDB exists per context, and it lives as long as the life of the context. Is this true?
2. It looks like NATable objects might be created on the statement heap or on the context heap. It looks like both may be cached. Is this true?
3. Assuming "yes" on question 2, how do the ones created on the statement heap get removed when the statement heap itself is destroyed?
Thanks,
Dave
RE: NATableDB (NATable cache)
Posted by Dave Birdsall <da...@esgyn.com>.
And as soon as I hit "send", I think I found some answers (to questions 2 and 3 anyway).
In the second NATableDB::get function, right after inserting an NATable object in the cache, there is logic like this:
//insert into cache
insert(table); <-- sticks an object in the cache, whether or not it is "cacheable"
//if we are using the cache
//if this NATable object is cacheable
if((useCache_) &&
(corrName.isCacheable()))
{
... do a bunch of stuff such as maintain high water marks ...
}
else{
//this has to be on the context heap since we need
//it after the statement heap has been remove
ExtendedQualName * nonCacheableTableName = new(CmpCommon::contextHeap())
ExtendedQualName(corrName.getExtendedQualNameObj(),
CmpCommon::contextHeap());
//insert into list of names of special tables
nonCacheableTableList_.insert(nonCacheableTableName);
// insert into list of non cacheable table idents. This
// allows the idents to be removed after the statement so
// the context heap doesn't keep growing.
const LIST(CollIndex) & tableIdList = table->getTableIdList();
for(CollIndex i = 0; i < tableIdList.entries(); i++)
{
nonCacheableTableIdents_.insert(tableIdList[i]);
}
}
So, the answer is, everything gets cached. But after the statement heap is destroyed, we come back to NATableDB somehow and remove all the entries that were allocated from the statement heap. For this to work correctly, however, this invariant relation must be maintained: (NATable was allocated on statement heap) if and only if (((useCache_) && (corrName.isCacheable()))
(Maybe it would be simpler and more robust to just check which heap the NATable object was allocated on instead.)
Thanks,
Dave
From: Dave Birdsall
Sent: Thursday, June 8, 2017 11:21 AM
To: dev@trafodion.incubator.apache.org
Subject: NATableDB (NATable cache)
Hi Trafodion developers,
I'm debugging some issues pertaining to NATable descriptors, and need to understand the behavior of the NATable cache (that is, NATableDB). I can figure it out from the code, but it might be faster to pick the collective Trafodion brains.
Some questions about NATableDB:
1. I have the impression that one NATableDB exists per context, and it lives as long as the life of the context. Is this true?
2. It looks like NATable objects might be created on the statement heap or on the context heap. It looks like both may be cached. Is this true?
3. Assuming "yes" on question 2, how do the ones created on the statement heap get removed when the statement heap itself is destroyed?
Thanks,
Dave