You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Erik Holstad (JIRA)" <ji...@apache.org> on 2009/02/07 03:44:59 UTC
[jira] Created: (HBASE-1189) Changing the map type used internally
for HbaseMapWritable.
Changing the map type used internally for HbaseMapWritable.
-------------------------------------------------------------
Key: HBASE-1189
URL: https://issues.apache.org/jira/browse/HBASE-1189
Project: Hadoop HBase
Issue Type: New Feature
Components: io
Reporter: Erik Holstad
Priority: Minor
For the CellCache the need for a HbaseMapWritable that has a different kind of map used internally
has showed up. The problem is that the instantiation of the map is done static so you have no control
over it and if extending HMW you get both the memory usage for the parent as well as the child.
After trying out different ideas on how to solve this, it seems like the easiest way would be to have a setMapType
method and just set it to null in the code. If the old code would be kept you would need to instantiate 2
different maps. The problem with setting it to null is that all the old code needs to be changed to fit the new model,
and also be used in the future.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1189) Changing the map type used internally
for HbaseMapWritable.
Posted by "Erik Holstad (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Erik Holstad updated HBASE-1189:
--------------------------------
Attachment: (was: HBASE-1189-v2.patch)
> Changing the map type used internally for HbaseMapWritable.
> -------------------------------------------------------------
>
> Key: HBASE-1189
> URL: https://issues.apache.org/jira/browse/HBASE-1189
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: io
> Affects Versions: 0.20.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Attachments: HBASE-1189-v1.patch, HBASE-1189-v2.patch, HbaseMapWritable.java, StaticLoader.java
>
>
> For the CellCache the need for a HbaseMapWritable that has a different kind of map used internally
> has showed up. The problem is that the instantiation of the map is done static so you have no control
> over it and if extending HMW you get both the memory usage for the parent as well as the child.
> After trying out different ideas on how to solve this, it seems like the easiest way would be to have a setMapType
> method and just set it to null in the code. If the old code would be kept you would need to instantiate 2
> different maps. The problem with setting it to null is that all the old code needs to be changed to fit the new model,
> and also be used in the future.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1189) Changing the map type used
internally for HbaseMapWritable.
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12671518#action_12671518 ]
stack commented on HBASE-1189:
------------------------------
Hey Erik. Which map are we talking of? Is it CODE_TO_CLASS (and its inverse) or is it
{code}
private SortedMap<byte [], V> instance =
new TreeMap<byte [], V>(Bytes.BYTES_COMPARATOR);
{code}
If the former, thats once for all instances of HBW. If latter, you want to just something other than TreeMap?
If you subclass, you think there much overhead? 8 bytes or so? You trying to be parsimonious? Good stuff.
> Changing the map type used internally for HbaseMapWritable.
> -------------------------------------------------------------
>
> Key: HBASE-1189
> URL: https://issues.apache.org/jira/browse/HBASE-1189
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: io
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
>
> For the CellCache the need for a HbaseMapWritable that has a different kind of map used internally
> has showed up. The problem is that the instantiation of the map is done static so you have no control
> over it and if extending HMW you get both the memory usage for the parent as well as the child.
> After trying out different ideas on how to solve this, it seems like the easiest way would be to have a setMapType
> method and just set it to null in the code. If the old code would be kept you would need to instantiate 2
> different maps. The problem with setting it to null is that all the old code needs to be changed to fit the new model,
> and also be used in the future.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1189) Changing the map type used internally
for HbaseMapWritable.
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-1189:
-------------------------
Resolution: Fixed
Fix Version/s: 0.20.0
Status: Resolved (was: Patch Available)
Committed. Thanks for the patch Erik.
> Changing the map type used internally for HbaseMapWritable.
> -------------------------------------------------------------
>
> Key: HBASE-1189
> URL: https://issues.apache.org/jira/browse/HBASE-1189
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: io
> Affects Versions: 0.20.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 0.20.0
>
> Attachments: HBASE-1189-v1.patch, HBASE-1189-v2.patch, HbaseMapWritable.java, StaticLoader.java
>
>
> For the CellCache the need for a HbaseMapWritable that has a different kind of map used internally
> has showed up. The problem is that the instantiation of the map is done static so you have no control
> over it and if extending HMW you get both the memory usage for the parent as well as the child.
> After trying out different ideas on how to solve this, it seems like the easiest way would be to have a setMapType
> method and just set it to null in the code. If the old code would be kept you would need to instantiate 2
> different maps. The problem with setting it to null is that all the old code needs to be changed to fit the new model,
> and also be used in the future.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1189) Changing the map type used internally
for HbaseMapWritable.
Posted by "Erik Holstad (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Erik Holstad updated HBASE-1189:
--------------------------------
Attachment: StaticLoader.java
HbaseMapWritable.java
A new version of the HbaseMapWritable that does not have a static constructor but implements an
interface that takes care of those static tasks.
> Changing the map type used internally for HbaseMapWritable.
> -------------------------------------------------------------
>
> Key: HBASE-1189
> URL: https://issues.apache.org/jira/browse/HBASE-1189
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: io
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Attachments: HbaseMapWritable.java, StaticLoader.java
>
>
> For the CellCache the need for a HbaseMapWritable that has a different kind of map used internally
> has showed up. The problem is that the instantiation of the map is done static so you have no control
> over it and if extending HMW you get both the memory usage for the parent as well as the child.
> After trying out different ideas on how to solve this, it seems like the easiest way would be to have a setMapType
> method and just set it to null in the code. If the old code would be kept you would need to instantiate 2
> different maps. The problem with setting it to null is that all the old code needs to be changed to fit the new model,
> and also be used in the future.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1189) Changing the map type used internally
for HbaseMapWritable.
Posted by "Erik Holstad (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Erik Holstad updated HBASE-1189:
--------------------------------
Attachment: HBASE-1189-v2.patch
New patch with some small fixes, changed the name of the static loader to be
better and shorter.
> Changing the map type used internally for HbaseMapWritable.
> -------------------------------------------------------------
>
> Key: HBASE-1189
> URL: https://issues.apache.org/jira/browse/HBASE-1189
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: io
> Affects Versions: 0.20.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Attachments: HBASE-1189-v1.patch, HBASE-1189-v2.patch, HbaseMapWritable.java, StaticLoader.java
>
>
> For the CellCache the need for a HbaseMapWritable that has a different kind of map used internally
> has showed up. The problem is that the instantiation of the map is done static so you have no control
> over it and if extending HMW you get both the memory usage for the parent as well as the child.
> After trying out different ideas on how to solve this, it seems like the easiest way would be to have a setMapType
> method and just set it to null in the code. If the old code would be kept you would need to instantiate 2
> different maps. The problem with setting it to null is that all the old code needs to be changed to fit the new model,
> and also be used in the future.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HBASE-1189) Changing the map type used internally
for HbaseMapWritable.
Posted by "Erik Holstad (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Erik Holstad reassigned HBASE-1189:
-----------------------------------
Assignee: Erik Holstad
> Changing the map type used internally for HbaseMapWritable.
> -------------------------------------------------------------
>
> Key: HBASE-1189
> URL: https://issues.apache.org/jira/browse/HBASE-1189
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: io
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
>
> For the CellCache the need for a HbaseMapWritable that has a different kind of map used internally
> has showed up. The problem is that the instantiation of the map is done static so you have no control
> over it and if extending HMW you get both the memory usage for the parent as well as the child.
> After trying out different ideas on how to solve this, it seems like the easiest way would be to have a setMapType
> method and just set it to null in the code. If the old code would be kept you would need to instantiate 2
> different maps. The problem with setting it to null is that all the old code needs to be changed to fit the new model,
> and also be used in the future.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HBASE-1189) Changing the map type
used internally for HbaseMapWritable.
Posted by "Erik Holstad (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12675090#action_12675090 ]
erikholstad@gmail.com edited comment on HBASE-1189 at 2/19/09 11:07 AM:
---------------------------------------------------------------
New patch with some small fixes, changed the name of the static loader to be
better and shorter.
I think that the variables in the class needs to be public, tried to make them private
though it would make more sense but the complier didn't like that.
Not sure what will happened if you have the internal methods set to static too, but I would
assume that they would take up memory.
Yes was thinking that you edit the file if you want to add more classes, just like you did in
the past, not really sure how you would do it otherwise since it is static.
was (Author: erikholstad@gmail.com):
New patch with some small fixes, changed the name of the static loader to be
better and shorter.
> Changing the map type used internally for HbaseMapWritable.
> -------------------------------------------------------------
>
> Key: HBASE-1189
> URL: https://issues.apache.org/jira/browse/HBASE-1189
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: io
> Affects Versions: 0.20.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Attachments: HBASE-1189-v1.patch, HBASE-1189-v2.patch, HbaseMapWritable.java, StaticLoader.java
>
>
> For the CellCache the need for a HbaseMapWritable that has a different kind of map used internally
> has showed up. The problem is that the instantiation of the map is done static so you have no control
> over it and if extending HMW you get both the memory usage for the parent as well as the child.
> After trying out different ideas on how to solve this, it seems like the easiest way would be to have a setMapType
> method and just set it to null in the code. If the old code would be kept you would need to instantiate 2
> different maps. The problem with setting it to null is that all the old code needs to be changed to fit the new model,
> and also be used in the future.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1189) Changing the map type used
internally for HbaseMapWritable.
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12675086#action_12675086 ]
stack commented on HBASE-1189:
------------------------------
Patch looks great.
Would suggest giving the interface a name that is less of a mouthful. Rather than StaticHBaseMapWritableLoader, how about CodeToClass (or CodeToClassAndBack)?
Do these have to be public Erik?
{code}
+ static final Map<Byte, Class<?>> CODE_TO_CLASS =
+ new HashMap<Byte, Class<?>>();
{code}
What if the client class is not in same package?
Does this need to be public since its only used inside this interface?
{code}+ public class InternalStaticLoader{
{code}
Should it be static too as in + public static class InternalStaticLoader{ (It might not work).
Should this be private too since internal to this Interface?
{code}
+ Class[] classList = {byte[].class, Cell.class};
{code}
To add classes, I edit the code? (Which is what we've done up to this).
Its interesting that we don't use it with HStoreKey nor ImmutableBytes anymore (I think thats probably good).
Good stuff Erik.
> Changing the map type used internally for HbaseMapWritable.
> -------------------------------------------------------------
>
> Key: HBASE-1189
> URL: https://issues.apache.org/jira/browse/HBASE-1189
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: io
> Affects Versions: 0.20.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Attachments: HBASE-1189-v1.patch, HbaseMapWritable.java, StaticLoader.java
>
>
> For the CellCache the need for a HbaseMapWritable that has a different kind of map used internally
> has showed up. The problem is that the instantiation of the map is done static so you have no control
> over it and if extending HMW you get both the memory usage for the parent as well as the child.
> After trying out different ideas on how to solve this, it seems like the easiest way would be to have a setMapType
> method and just set it to null in the code. If the old code would be kept you would need to instantiate 2
> different maps. The problem with setting it to null is that all the old code needs to be changed to fit the new model,
> and also be used in the future.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1189) Changing the map type used internally
for HbaseMapWritable.
Posted by "Erik Holstad (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Erik Holstad updated HBASE-1189:
--------------------------------
Affects Version/s: 0.20.0
Status: Patch Available (was: Open)
Updated the HbaseMapWritable to not use a static constructor but instead
implement a static interface.
> Changing the map type used internally for HbaseMapWritable.
> -------------------------------------------------------------
>
> Key: HBASE-1189
> URL: https://issues.apache.org/jira/browse/HBASE-1189
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: io
> Affects Versions: 0.20.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Attachments: HBASE-1189-v1.patch, HbaseMapWritable.java, StaticLoader.java
>
>
> For the CellCache the need for a HbaseMapWritable that has a different kind of map used internally
> has showed up. The problem is that the instantiation of the map is done static so you have no control
> over it and if extending HMW you get both the memory usage for the parent as well as the child.
> After trying out different ideas on how to solve this, it seems like the easiest way would be to have a setMapType
> method and just set it to null in the code. If the old code would be kept you would need to instantiate 2
> different maps. The problem with setting it to null is that all the old code needs to be changed to fit the new model,
> and also be used in the future.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1189) Changing the map type used internally
for HbaseMapWritable.
Posted by "Erik Holstad (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Erik Holstad updated HBASE-1189:
--------------------------------
Attachment: HBASE-1189-v1.patch
> Changing the map type used internally for HbaseMapWritable.
> -------------------------------------------------------------
>
> Key: HBASE-1189
> URL: https://issues.apache.org/jira/browse/HBASE-1189
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: io
> Affects Versions: 0.20.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Attachments: HBASE-1189-v1.patch, HbaseMapWritable.java, StaticLoader.java
>
>
> For the CellCache the need for a HbaseMapWritable that has a different kind of map used internally
> has showed up. The problem is that the instantiation of the map is done static so you have no control
> over it and if extending HMW you get both the memory usage for the parent as well as the child.
> After trying out different ideas on how to solve this, it seems like the easiest way would be to have a setMapType
> method and just set it to null in the code. If the old code would be kept you would need to instantiate 2
> different maps. The problem with setting it to null is that all the old code needs to be changed to fit the new model,
> and also be used in the future.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1189) Changing the map type used internally
for HbaseMapWritable.
Posted by "Erik Holstad (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Erik Holstad updated HBASE-1189:
--------------------------------
Attachment: HBASE-1189-v2.patch
> Changing the map type used internally for HbaseMapWritable.
> -------------------------------------------------------------
>
> Key: HBASE-1189
> URL: https://issues.apache.org/jira/browse/HBASE-1189
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: io
> Affects Versions: 0.20.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Attachments: HBASE-1189-v1.patch, HBASE-1189-v2.patch, HbaseMapWritable.java, StaticLoader.java
>
>
> For the CellCache the need for a HbaseMapWritable that has a different kind of map used internally
> has showed up. The problem is that the instantiation of the map is done static so you have no control
> over it and if extending HMW you get both the memory usage for the parent as well as the child.
> After trying out different ideas on how to solve this, it seems like the easiest way would be to have a setMapType
> method and just set it to null in the code. If the old code would be kept you would need to instantiate 2
> different maps. The problem with setting it to null is that all the old code needs to be changed to fit the new model,
> and also be used in the future.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.