You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org> on 2010/09/17 00:52:32 UTC
[jira] Created: (HBASE-3008) Memstore.updateColumnValue passes
wrong flag to heapSizeChange
Memstore.updateColumnValue passes wrong flag to heapSizeChange
--------------------------------------------------------------
Key: HBASE-3008
URL: https://issues.apache.org/jira/browse/HBASE-3008
Project: HBase
Issue Type: Bug
Reporter: Jean-Daniel Cryans
Assignee: Jean-Daniel Cryans
Fix For: 0.90.0
Memstore.updateColumnValue passes the wrong flag to heapSizeChange, making it so that the size keeps growing while the actual size is probably the same. For example, in our production environment we see tables that only take ICVs doing flushes of a few KBs when it thinks there's 64MB in:
{noformat}
Started memstore flush for region somecountingtable,,1248719998664.121282795. Current region memstore size 64.0
Added hdfs://borg9:9000/hbase/somecountingtable/121282795/counter/3564459650504019443, entries=905, sequenceid=72504291507,
memsize=183.3k, filesize=18.5k to somecountingtable,,1248719998664.121282795
{noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3008) Memstore.updateColumnValue passes
wrong flag to heapSizeChange
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12911036#action_12911036 ]
stack commented on HBASE-3008:
------------------------------
+1
> Memstore.updateColumnValue passes wrong flag to heapSizeChange
> --------------------------------------------------------------
>
> Key: HBASE-3008
> URL: https://issues.apache.org/jira/browse/HBASE-3008
> Project: HBase
> Issue Type: Bug
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.90.0
>
> Attachments: HBASE-3008.patch
>
>
> Memstore.updateColumnValue passes the wrong flag to heapSizeChange, making it so that the size keeps growing while the actual size is probably the same. For example, in our production environment we see tables that only take ICVs doing flushes of a few KBs when it thinks there's 64MB in:
> {noformat}
> Started memstore flush for region somecountingtable,,1248719998664.121282795. Current region memstore size 64.0
> Added hdfs://borg9:9000/hbase/somecountingtable/121282795/counter/3564459650504019443, entries=905, sequenceid=72504291507,
> memsize=183.3k, filesize=18.5k to somecountingtable,,1248719998664.121282795
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3008) Memstore.updateColumnValue passes
wrong flag to heapSizeChange
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12911059#action_12911059 ]
Todd Lipcon commented on HBASE-3008:
------------------------------------
Unit test? This seems pretty easy to test.
> Memstore.updateColumnValue passes wrong flag to heapSizeChange
> --------------------------------------------------------------
>
> Key: HBASE-3008
> URL: https://issues.apache.org/jira/browse/HBASE-3008
> Project: HBase
> Issue Type: Bug
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.90.0
>
> Attachments: HBASE-3008.patch
>
>
> Memstore.updateColumnValue passes the wrong flag to heapSizeChange, making it so that the size keeps growing while the actual size is probably the same. For example, in our production environment we see tables that only take ICVs doing flushes of a few KBs when it thinks there's 64MB in:
> {noformat}
> Started memstore flush for region somecountingtable,,1248719998664.121282795. Current region memstore size 64.0
> Added hdfs://borg9:9000/hbase/somecountingtable/121282795/counter/3564459650504019443, entries=905, sequenceid=72504291507,
> memsize=183.3k, filesize=18.5k to somecountingtable,,1248719998664.121282795
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-3008) Memstore.updateColumnValue passes
wrong flag to heapSizeChange
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans updated HBASE-3008:
--------------------------------------
Fix Version/s: 0.89.20100924
I'm putting this in the next 0.89 without a test, will write test later for 0.90
> Memstore.updateColumnValue passes wrong flag to heapSizeChange
> --------------------------------------------------------------
>
> Key: HBASE-3008
> URL: https://issues.apache.org/jira/browse/HBASE-3008
> Project: HBase
> Issue Type: Bug
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.89.20100924, 0.90.0
>
> Attachments: HBASE-3008.patch
>
>
> Memstore.updateColumnValue passes the wrong flag to heapSizeChange, making it so that the size keeps growing while the actual size is probably the same. For example, in our production environment we see tables that only take ICVs doing flushes of a few KBs when it thinks there's 64MB in:
> {noformat}
> Started memstore flush for region somecountingtable,,1248719998664.121282795. Current region memstore size 64.0
> Added hdfs://borg9:9000/hbase/somecountingtable/121282795/counter/3564459650504019443, entries=905, sequenceid=72504291507,
> memsize=183.3k, filesize=18.5k to somecountingtable,,1248719998664.121282795
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-3008) Memstore.updateColumnValue passes
wrong flag to heapSizeChange
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans updated HBASE-3008:
--------------------------------------
Attachment: HBASE-3008.patch
This is the patch that Ryan wrote and that fixes the issue.
> Memstore.updateColumnValue passes wrong flag to heapSizeChange
> --------------------------------------------------------------
>
> Key: HBASE-3008
> URL: https://issues.apache.org/jira/browse/HBASE-3008
> Project: HBase
> Issue Type: Bug
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.90.0
>
> Attachments: HBASE-3008.patch
>
>
> Memstore.updateColumnValue passes the wrong flag to heapSizeChange, making it so that the size keeps growing while the actual size is probably the same. For example, in our production environment we see tables that only take ICVs doing flushes of a few KBs when it thinks there's 64MB in:
> {noformat}
> Started memstore flush for region somecountingtable,,1248719998664.121282795. Current region memstore size 64.0
> Added hdfs://borg9:9000/hbase/somecountingtable/121282795/counter/3564459650504019443, entries=905, sequenceid=72504291507,
> memsize=183.3k, filesize=18.5k to somecountingtable,,1248719998664.121282795
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-3008) Memstore.updateColumnValue passes
wrong flag to heapSizeChange
Posted by "ryan rawson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ryan rawson resolved HBASE-3008.
--------------------------------
Resolution: Fixed
this was commited as part of r1000276 as well as r1005261
> Memstore.updateColumnValue passes wrong flag to heapSizeChange
> --------------------------------------------------------------
>
> Key: HBASE-3008
> URL: https://issues.apache.org/jira/browse/HBASE-3008
> Project: HBase
> Issue Type: Bug
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.89.20100924, 0.90.0
>
> Attachments: HBASE-3008.patch
>
>
> Memstore.updateColumnValue passes the wrong flag to heapSizeChange, making it so that the size keeps growing while the actual size is probably the same. For example, in our production environment we see tables that only take ICVs doing flushes of a few KBs when it thinks there's 64MB in:
> {noformat}
> Started memstore flush for region somecountingtable,,1248719998664.121282795. Current region memstore size 64.0
> Added hdfs://borg9:9000/hbase/somecountingtable/121282795/counter/3564459650504019443, entries=905, sequenceid=72504291507,
> memsize=183.3k, filesize=18.5k to somecountingtable,,1248719998664.121282795
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3008) Memstore.updateColumnValue passes
wrong flag to heapSizeChange
Posted by "HBase Review Board (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12918380#action_12918380 ]
HBase Review Board commented on HBASE-3008:
-------------------------------------------
Message from: "Ryan Rawson" <ry...@gmail.com>
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/965/
-----------------------------------------------------------
Review request for hbase.
Summary
-------
a problem we observed with HBASE-3008 + HBASE-2997, that memstore sizes can go negative preventing us from flushing them even if there is data within. This is a data loss bug.
This addresses bug HBASE-3008.
http://issues.apache.org/jira/browse/HBASE-3008
Diffs
-----
trunk/CHANGES.txt 1004867
trunk/src/main/java/org/apache/hadoop/hbase/KeyValue.java 1004867
trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java 1004867
trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java 1004867
Diff: http://review.cloudera.org/r/965/diff
Testing
-------
Thanks,
Ryan
> Memstore.updateColumnValue passes wrong flag to heapSizeChange
> --------------------------------------------------------------
>
> Key: HBASE-3008
> URL: https://issues.apache.org/jira/browse/HBASE-3008
> Project: HBase
> Issue Type: Bug
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.89.20100924, 0.90.0
>
> Attachments: HBASE-3008.patch
>
>
> Memstore.updateColumnValue passes the wrong flag to heapSizeChange, making it so that the size keeps growing while the actual size is probably the same. For example, in our production environment we see tables that only take ICVs doing flushes of a few KBs when it thinks there's 64MB in:
> {noformat}
> Started memstore flush for region somecountingtable,,1248719998664.121282795. Current region memstore size 64.0
> Added hdfs://borg9:9000/hbase/somecountingtable/121282795/counter/3564459650504019443, entries=905, sequenceid=72504291507,
> memsize=183.3k, filesize=18.5k to somecountingtable,,1248719998664.121282795
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.