You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org> on 2008/10/31 17:59:44 UTC
[jira] Created: (HBASE-975) Improve MapFile performance for start
and end key
Improve MapFile performance for start and end key
-------------------------------------------------
Key: HBASE-975
URL: https://issues.apache.org/jira/browse/HBASE-975
Project: Hadoop HBase
Issue Type: Improvement
Reporter: Jean-Daniel Cryans
Attachments: hbase-975.patch
Keeping a MapFile's start and end key in cache would save us some seeks, see if it can be done.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-975) Improve MapFile performance for start
and end key
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack resolved HBASE-975.
-------------------------
Resolution: Cannot Reproduce
Committed.
Primitive testing shows that we might actually be doing MORE seeks now with this patch in place (e.g. we open new Readers when we go to compact so we seek for for start and final keys we never use -- we also load in their indices for a second time so there is extra memory pressure during compactions). Let me test some more. I might back out the core of the patch that does the read of first and last keys but will leave the rest of the cleanup refactoring. Also committing 'cos Jim wants to check his HBASE-964 fix.
> Improve MapFile performance for start and end key
> -------------------------------------------------
>
> Key: HBASE-975
> URL: https://issues.apache.org/jira/browse/HBASE-975
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: stack
> Fix For: 0.19.0
>
> Attachments: 975-v2.patch, 975-v4.patch, hbase-975.patch
>
>
> Keeping a MapFile's start and end key in cache would save us some seeks, see if it can be done.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-975) Improve MapFile performance for start
and end key
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-975:
------------------------
Attachment: 975-v4.patch
This patch is basically working. Uses cached start and end keys to avoid seeking. Also moves all of the MapFile subclasses -- bloom, half and hbase -- out into io package.
> Improve MapFile performance for start and end key
> -------------------------------------------------
>
> Key: HBASE-975
> URL: https://issues.apache.org/jira/browse/HBASE-975
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: stack
> Fix For: 0.19.0
>
> Attachments: 975-v2.patch, 975-v4.patch, hbase-975.patch
>
>
> Keeping a MapFile's start and end key in cache would save us some seeks, see if it can be done.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Reopened: (HBASE-975) Improve MapFile performance for start
and end key
Posted by "Jim Kellerman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jim Kellerman reopened HBASE-975:
---------------------------------
Reopening issue to change resolution status.
> Improve MapFile performance for start and end key
> -------------------------------------------------
>
> Key: HBASE-975
> URL: https://issues.apache.org/jira/browse/HBASE-975
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: stack
> Fix For: 0.19.0
>
> Attachments: 975-v2.patch, 975-v4.patch, hbase-975.patch
>
>
> Keeping a MapFile's start and end key in cache would save us some seeks, see if it can be done.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-975) Improve MapFile performance for start
and end key
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-975:
------------------------
Fix Version/s: 0.19.0
See if can be done for 0.19.0 (Looking at splitting, everytime we want to figure if we're to split, we seek around in mapfile to find start and end keys).
> Improve MapFile performance for start and end key
> -------------------------------------------------
>
> Key: HBASE-975
> URL: https://issues.apache.org/jira/browse/HBASE-975
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Fix For: 0.19.0
>
> Attachments: hbase-975.patch
>
>
> Keeping a MapFile's start and end key in cache would save us some seeks, see if it can be done.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HBASE-975) Improve MapFile performance for start
and end key
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack reassigned HBASE-975:
---------------------------
Assignee: stack
> Improve MapFile performance for start and end key
> -------------------------------------------------
>
> Key: HBASE-975
> URL: https://issues.apache.org/jira/browse/HBASE-975
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: stack
> Fix For: 0.19.0
>
> Attachments: hbase-975.patch
>
>
> Keeping a MapFile's start and end key in cache would save us some seeks, see if it can be done.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-975) Improve MapFile performance for start
and end key
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-975:
------------------------
Attachment: 975-v2.patch
Adds caching of first and last keys to HbaseMapFile.HbaseReader and to HalfReader. Use cached keys whenever they're asked for. Added refactoring that cleans up some long methods. Added tests of new HbaseReader.contains method.
Todo is using contains method inside all get methods, etc.
> Improve MapFile performance for start and end key
> -------------------------------------------------
>
> Key: HBASE-975
> URL: https://issues.apache.org/jira/browse/HBASE-975
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: stack
> Fix For: 0.19.0
>
> Attachments: 975-v2.patch, hbase-975.patch
>
>
> Keeping a MapFile's start and end key in cache would save us some seeks, see if it can be done.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-975) Improve MapFile performance for start
and end key
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans updated HBASE-975:
-------------------------------------
Attachment: hbase-975.patch
Part of the job is done but for some reason it doesn't seem split-friendly.
> Improve MapFile performance for start and end key
> -------------------------------------------------
>
> Key: HBASE-975
> URL: https://issues.apache.org/jira/browse/HBASE-975
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Attachments: hbase-975.patch
>
>
> Keeping a MapFile's start and end key in cache would save us some seeks, see if it can be done.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-975) Improve MapFile performance for start
and end key
Posted by "Jim Kellerman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jim Kellerman resolved HBASE-975.
---------------------------------
Resolution: Won't Fix
As HBASE-980 undid the caching of start and end rows, resolving this issue as won't fix.
> Improve MapFile performance for start and end key
> -------------------------------------------------
>
> Key: HBASE-975
> URL: https://issues.apache.org/jira/browse/HBASE-975
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: stack
> Fix For: 0.19.0
>
> Attachments: 975-v2.patch, 975-v4.patch, hbase-975.patch
>
>
> Keeping a MapFile's start and end key in cache would save us some seeks, see if it can be done.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-975) Improve MapFile performance for start
and end key
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12644871#action_12644871 ]
stack commented on HBASE-975:
-----------------------------
I resolved it as non-reproducible which is wrong but I can't seem to fix it.
Also, this commit may fail up on hudson in TestSplit but HBASE-964 work is supposed to fix it (The last commit on HBASE-964 does not seem to have fixed it -- checking).
> Improve MapFile performance for start and end key
> -------------------------------------------------
>
> Key: HBASE-975
> URL: https://issues.apache.org/jira/browse/HBASE-975
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: stack
> Fix For: 0.19.0
>
> Attachments: 975-v2.patch, 975-v4.patch, hbase-975.patch
>
>
> Keeping a MapFile's start and end key in cache would save us some seeks, see if it can be done.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.