You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "stack (JIRA)" <ji...@apache.org> on 2009/02/25 06:47:04 UTC
[jira] Created: (HBASE-1218) Implement in-memory column
Implement in-memory column
--------------------------
Key: HBASE-1218
URL: https://issues.apache.org/jira/browse/HBASE-1218
Project: Hadoop HBase
Issue Type: New Feature
Reporter: stack
Fix For: 0.20.0
HCD already talks of in-memory columns; its just not implemented. With hfile this should now be possible -- if set, read in the whole storefile and serve from the blockbuffer.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HBASE-1218) Implement in-memory column
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Gray reassigned HBASE-1218:
------------------------------------
Assignee: Jonathan Gray (was: ryan rawson)
> Implement in-memory column
> --------------------------
>
> Key: HBASE-1218
> URL: https://issues.apache.org/jira/browse/HBASE-1218
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: stack
> Assignee: Jonathan Gray
> Fix For: 0.20.0
>
>
> HCD already talks of in-memory columns; its just not implemented. With hfile this should now be possible -- if set, read in the whole storefile and serve from the blockbuffer.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HBASE-1218) Implement in-memory column
Posted by "ryan rawson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ryan rawson reassigned HBASE-1218:
----------------------------------
Assignee: ryan rawson (was: Jonathan Gray)
> Implement in-memory column
> --------------------------
>
> Key: HBASE-1218
> URL: https://issues.apache.org/jira/browse/HBASE-1218
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: stack
> Assignee: ryan rawson
> Fix For: 0.20.0
>
>
> HCD already talks of in-memory columns; its just not implemented. With hfile this should now be possible -- if set, read in the whole storefile and serve from the blockbuffer.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1218) Implement in-memory column
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12725856#action_12725856 ]
Jonathan Gray commented on HBASE-1218:
--------------------------------------
The LRU in trunk now supports in-memory blocks.
In HFile.Reader.readBlock(int) we need to change cache.cacheBlock(blockName, blockBuf) to cache.cacheBlock(blockName, blockBuf, boolean inMemory).
HFile.Reader gets instantiated from within a Store so should be easy enough to pass something in from the family configuration there.
I strongly recommend lazy-loading, no pre-fetching of blocks. That's how they do it in big table. If you want to warm the blocks, just scan the table.
> Implement in-memory column
> --------------------------
>
> Key: HBASE-1218
> URL: https://issues.apache.org/jira/browse/HBASE-1218
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: stack
> Assignee: ryan rawson
> Fix For: 0.20.0
>
>
> HCD already talks of in-memory columns; its just not implemented. With hfile this should now be possible -- if set, read in the whole storefile and serve from the blockbuffer.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HBASE-1218) Implement in-memory column
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Gray reassigned HBASE-1218:
------------------------------------
Assignee: Jonathan Gray
> Implement in-memory column
> --------------------------
>
> Key: HBASE-1218
> URL: https://issues.apache.org/jira/browse/HBASE-1218
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: stack
> Assignee: Jonathan Gray
> Fix For: 0.20.0
>
>
> HCD already talks of in-memory columns; its just not implemented. With hfile this should now be possible -- if set, read in the whole storefile and serve from the blockbuffer.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1218) Implement in-memory column
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Gray updated HBASE-1218:
---------------------------------
Attachment: HBASE-1218-v2.patch
post 1597 patch
> Implement in-memory column
> --------------------------
>
> Key: HBASE-1218
> URL: https://issues.apache.org/jira/browse/HBASE-1218
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: stack
> Assignee: Jonathan Gray
> Fix For: 0.20.0
>
> Attachments: HBASE-1218-v1.patch, HBASE-1218-v2.patch
>
>
> HCD already talks of in-memory columns; its just not implemented. With hfile this should now be possible -- if set, read in the whole storefile and serve from the blockbuffer.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1218) Implement in-memory column
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-1218:
-------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
Committed. Thanks for patch Jon.
> Implement in-memory column
> --------------------------
>
> Key: HBASE-1218
> URL: https://issues.apache.org/jira/browse/HBASE-1218
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: stack
> Assignee: Jonathan Gray
> Fix For: 0.20.0
>
> Attachments: HBASE-1218-v1.patch, HBASE-1218-v2.patch, HBASE-1218-v3.patch
>
>
> HCD already talks of in-memory columns; its just not implemented. With hfile this should now be possible -- if set, read in the whole storefile and serve from the blockbuffer.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1218) Implement in-memory column
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Gray updated HBASE-1218:
---------------------------------
Attachment: HBASE-1218-v1.patch
Passes a boolean in from Store to StoreFile to HFile.Reader and then to the LRU block cache.
Not much to it.
This patch incorporates some of the changes from HBASE-1597. I will remake a patch after that gets committed, but this is up for review.
> Implement in-memory column
> --------------------------
>
> Key: HBASE-1218
> URL: https://issues.apache.org/jira/browse/HBASE-1218
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: stack
> Assignee: Jonathan Gray
> Fix For: 0.20.0
>
> Attachments: HBASE-1218-v1.patch
>
>
> HCD already talks of in-memory columns; its just not implemented. With hfile this should now be possible -- if set, read in the whole storefile and serve from the blockbuffer.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1218) Implement in-memory column
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Gray updated HBASE-1218:
---------------------------------
Attachment: HBASE-1218-v3.patch
Adds better logging at the end of evictions, outputting the sizes of the three priorities of blocks (single, multiple, memory)
> Implement in-memory column
> --------------------------
>
> Key: HBASE-1218
> URL: https://issues.apache.org/jira/browse/HBASE-1218
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: stack
> Assignee: Jonathan Gray
> Fix For: 0.20.0
>
> Attachments: HBASE-1218-v1.patch, HBASE-1218-v2.patch, HBASE-1218-v3.patch
>
>
> HCD already talks of in-memory columns; its just not implemented. With hfile this should now be possible -- if set, read in the whole storefile and serve from the blockbuffer.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1218) Implement in-memory column
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12727040#action_12727040 ]
stack commented on HBASE-1218:
------------------------------
Chatting with Jon, this patch needs more. In-memory is a table setting. Rare will be case where we want a table in memory. Jon is going to make it so in-memory is a column setting. I'll take care of migration part over in 1295.
> Implement in-memory column
> --------------------------
>
> Key: HBASE-1218
> URL: https://issues.apache.org/jira/browse/HBASE-1218
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: stack
> Assignee: Jonathan Gray
> Fix For: 0.20.0
>
> Attachments: HBASE-1218-v1.patch
>
>
> HCD already talks of in-memory columns; its just not implemented. With hfile this should now be possible -- if set, read in the whole storefile and serve from the blockbuffer.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1218) Implement in-memory column
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Gray updated HBASE-1218:
---------------------------------
Status: Patch Available (was: Open)
Ready for final review and commit.
> Implement in-memory column
> --------------------------
>
> Key: HBASE-1218
> URL: https://issues.apache.org/jira/browse/HBASE-1218
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: stack
> Assignee: Jonathan Gray
> Fix For: 0.20.0
>
> Attachments: HBASE-1218-v1.patch, HBASE-1218-v2.patch
>
>
> HCD already talks of in-memory columns; its just not implemented. With hfile this should now be possible -- if set, read in the whole storefile and serve from the blockbuffer.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.