You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Praveen Kumar (JIRA)" <ji...@apache.org> on 2011/07/02 02:35:28 UTC
[jira] [Created] (HBASE-4056) Support for using faster storage for
write-ahead log
Support for using faster storage for write-ahead log
----------------------------------------------------
Key: HBASE-4056
URL: https://issues.apache.org/jira/browse/HBASE-4056
Project: HBase
Issue Type: New Feature
Reporter: Praveen Kumar
Priority: Minor
On clusters with heterogeneous storage components like hard drives and flash memory, it could be beneficial to use flash memory for write-ahead log. This can be accomplished by using client side mount table support (HADOOP-7257) that is offered by HDFS federation (HDFS-1052) feature. One can define two HDFS namespaces (faster and slower), and configure HBase to use faster storage namespace for storing WAL.
This is an abstract task that captures the idea. More brainstorming and subtasks identification to follow.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4056) Support for using faster storage
for write-ahead log
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13061786#comment-13061786 ]
Jonathan Gray commented on HBASE-4056:
--------------------------------------
Thanks for opening this JIRA.
What do you see as the primary benefit of using flash for the WAL? I've seen some improvement in sequential write throughput, but not drastically different.
It seems to me that a significant benefit of using flash is the fast random read access, and there are no random reads on the WAL.
One idea that has floated around is to do something like cache-on-write to copy recently written files onto flash (in addition to HDFS) to allow for fast random read access. Or use flash as some kind of extension to the block cache.
But regardless, making all of this stuff configurable and supporting more diverse setups is a good thing in general. Some experiments and benchmarks around this would be awesome. Good stuff.
> Support for using faster storage for write-ahead log
> ----------------------------------------------------
>
> Key: HBASE-4056
> URL: https://issues.apache.org/jira/browse/HBASE-4056
> Project: HBase
> Issue Type: New Feature
> Reporter: Praveen Kumar
> Priority: Minor
> Labels: features
>
> On clusters with heterogeneous storage components like hard drives and flash memory, it could be beneficial to use flash memory for write-ahead log. This can be accomplished by using client side mount table support (HADOOP-7257) that is offered by HDFS federation (HDFS-1052) feature. One can define two HDFS namespaces (faster and slower), and configure HBase to use faster storage namespace for storing WAL.
> This is an abstract task that captures the idea. More brainstorming and subtasks identification to follow.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4056) Support for using faster storage
for write-ahead log
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13061672#comment-13061672 ]
stack commented on HBASE-4056:
------------------------------
Nice idea Praveen. We should just do this now. You can have one config refer to another so the wal fs can just refer to the root fs.
> Support for using faster storage for write-ahead log
> ----------------------------------------------------
>
> Key: HBASE-4056
> URL: https://issues.apache.org/jira/browse/HBASE-4056
> Project: HBase
> Issue Type: New Feature
> Reporter: Praveen Kumar
> Priority: Minor
> Labels: features
>
> On clusters with heterogeneous storage components like hard drives and flash memory, it could be beneficial to use flash memory for write-ahead log. This can be accomplished by using client side mount table support (HADOOP-7257) that is offered by HDFS federation (HDFS-1052) feature. One can define two HDFS namespaces (faster and slower), and configure HBase to use faster storage namespace for storing WAL.
> This is an abstract task that captures the idea. More brainstorming and subtasks identification to follow.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira