You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org> on 2009/08/25 20:33:59 UTC
[jira] Created: (HBASE-1792) [Regression] Cannot save timestamp in
the future
[Regression] Cannot save timestamp in the future
------------------------------------------------
Key: HBASE-1792
URL: https://issues.apache.org/jira/browse/HBASE-1792
Project: Hadoop HBase
Issue Type: Bug
Affects Versions: 0.20.0
Reporter: Jean-Daniel Cryans
0.20, compared to previous versions, doesn't let you save with a timestamp in the future and will set it to current time without telling you. This is really bad for users upgrading to 0.20 that were using those timestamps.
Example:
hbase(main):004:0> put 'testtable', 'r1', 'f1:c1', 'val', 5373965335336911168
0 row(s) in 0.0070 seconds
hbase(main):005:0> scan 'testtable'
ROW COLUMN+CELL
r1 column=f1:c1, timestamp=1251223892010, value=val
1 row(s) in 0.0380 seconds
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1792) [Regression] Cannot save timestamp in
the future
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans updated HBASE-1792:
--------------------------------------
Attachment: HBASE-1792.patch
Currently testing this patch.
> [Regression] Cannot save timestamp in the future
> ------------------------------------------------
>
> Key: HBASE-1792
> URL: https://issues.apache.org/jira/browse/HBASE-1792
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Jean-Daniel Cryans
> Attachments: HBASE-1792.patch
>
>
> 0.20, compared to previous versions, doesn't let you save with a timestamp in the future and will set it to current time without telling you. This is really bad for users upgrading to 0.20 that were using those timestamps.
> Example:
> hbase(main):004:0> put 'testtable', 'r1', 'f1:c1', 'val', 5373965335336911168
> 0 row(s) in 0.0070 seconds
> hbase(main):005:0> scan 'testtable'
> ROW COLUMN+CELL
> r1 column=f1:c1, timestamp=1251223892010, value=val
> 1 row(s) in 0.0380 seconds
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-1792) [Regression] Cannot save timestamp in
the future
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans resolved HBASE-1792.
---------------------------------------
Resolution: Fixed
Fix Version/s: 0.21.0
0.20.0
Assignee: Jean-Daniel Cryans
Hadoop Flags: [Reviewed]
Committed to branch and trunk, test passes and it fixes the issue.
> [Regression] Cannot save timestamp in the future
> ------------------------------------------------
>
> Key: HBASE-1792
> URL: https://issues.apache.org/jira/browse/HBASE-1792
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.20.0, 0.21.0
>
> Attachments: HBASE-1792.patch
>
>
> 0.20, compared to previous versions, doesn't let you save with a timestamp in the future and will set it to current time without telling you. This is really bad for users upgrading to 0.20 that were using those timestamps.
> Example:
> hbase(main):004:0> put 'testtable', 'r1', 'f1:c1', 'val', 5373965335336911168
> 0 row(s) in 0.0070 seconds
> hbase(main):005:0> scan 'testtable'
> ROW COLUMN+CELL
> r1 column=f1:c1, timestamp=1251223892010, value=val
> 1 row(s) in 0.0380 seconds
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1792) [Regression] Cannot save timestamp
in the future
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747618#action_12747618 ]
Jean-Daniel Cryans commented on HBASE-1792:
-------------------------------------------
Just so I don't forget it, the timestamp is replaced silently here in HRegion.updateKeys:
{code}
/**
* Checks if any stamps are > now. If so, sets them to now.
* <p>
* This acts to be prevent users from inserting future stamps as well as
* to replace LATEST_TIMESTAMP with now.
* @param keys
* @param now
* @return <code>true</code> when updating the time stamp completed.
*/
private boolean updateKeys(List<KeyValue> keys, byte [] now) {
if(keys == null || keys.isEmpty()) {
return false;
}
for(KeyValue key : keys) {
key.updateLatestStamp(now);
}
return true;
}
{code}
> [Regression] Cannot save timestamp in the future
> ------------------------------------------------
>
> Key: HBASE-1792
> URL: https://issues.apache.org/jira/browse/HBASE-1792
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Jean-Daniel Cryans
>
> 0.20, compared to previous versions, doesn't let you save with a timestamp in the future and will set it to current time without telling you. This is really bad for users upgrading to 0.20 that were using those timestamps.
> Example:
> hbase(main):004:0> put 'testtable', 'r1', 'f1:c1', 'val', 5373965335336911168
> 0 row(s) in 0.0070 seconds
> hbase(main):005:0> scan 'testtable'
> ROW COLUMN+CELL
> r1 column=f1:c1, timestamp=1251223892010, value=val
> 1 row(s) in 0.0380 seconds
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1792) [Regression] Cannot save timestamp
in the future
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747710#action_12747710 ]
Jonathan Gray commented on HBASE-1792:
--------------------------------------
+1 for commit on branch and trunk after your testing, jd.
> [Regression] Cannot save timestamp in the future
> ------------------------------------------------
>
> Key: HBASE-1792
> URL: https://issues.apache.org/jira/browse/HBASE-1792
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Jean-Daniel Cryans
> Attachments: HBASE-1792.patch
>
>
> 0.20, compared to previous versions, doesn't let you save with a timestamp in the future and will set it to current time without telling you. This is really bad for users upgrading to 0.20 that were using those timestamps.
> Example:
> hbase(main):004:0> put 'testtable', 'r1', 'f1:c1', 'val', 5373965335336911168
> 0 row(s) in 0.0070 seconds
> hbase(main):005:0> scan 'testtable'
> ROW COLUMN+CELL
> r1 column=f1:c1, timestamp=1251223892010, value=val
> 1 row(s) in 0.0380 seconds
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1792) [Regression] Cannot save timestamp
in the future
Posted by "ryan rawson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747625#action_12747625 ]
ryan rawson commented on HBASE-1792:
------------------------------------
we should move back to Long.MAX_VALUE as the thing to replace to 'now' i think...
The problem with deletes would be if the delete is set too far in the past or future, depending on the time range of the files that end up bracketing them. Solved with minor compaction, and works via scans.
> [Regression] Cannot save timestamp in the future
> ------------------------------------------------
>
> Key: HBASE-1792
> URL: https://issues.apache.org/jira/browse/HBASE-1792
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Jean-Daniel Cryans
>
> 0.20, compared to previous versions, doesn't let you save with a timestamp in the future and will set it to current time without telling you. This is really bad for users upgrading to 0.20 that were using those timestamps.
> Example:
> hbase(main):004:0> put 'testtable', 'r1', 'f1:c1', 'val', 5373965335336911168
> 0 row(s) in 0.0070 seconds
> hbase(main):005:0> scan 'testtable'
> ROW COLUMN+CELL
> r1 column=f1:c1, timestamp=1251223892010, value=val
> 1 row(s) in 0.0380 seconds
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1792) [Regression] Cannot save timestamp
in the future
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747575#action_12747575 ]
stack commented on HBASE-1792:
------------------------------
Yeah, I suppose old days we'd leave the ts alone and only change it to current time if it was HConstants.LATEST_TIMESTAMP (IIRC).
> [Regression] Cannot save timestamp in the future
> ------------------------------------------------
>
> Key: HBASE-1792
> URL: https://issues.apache.org/jira/browse/HBASE-1792
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Jean-Daniel Cryans
>
> 0.20, compared to previous versions, doesn't let you save with a timestamp in the future and will set it to current time without telling you. This is really bad for users upgrading to 0.20 that were using those timestamps.
> Example:
> hbase(main):004:0> put 'testtable', 'r1', 'f1:c1', 'val', 5373965335336911168
> 0 row(s) in 0.0070 seconds
> hbase(main):005:0> scan 'testtable'
> ROW COLUMN+CELL
> r1 column=f1:c1, timestamp=1251223892010, value=val
> 1 row(s) in 0.0380 seconds
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1792) [Regression] Cannot save timestamp
in the future
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747581#action_12747581 ]
Jonathan Gray commented on HBASE-1792:
--------------------------------------
There was a number of long discussions related to this. Unfortunately, I forgot where we ended up.
We tried to keep certain axioms, like "deletes only apply to older storefiles", but manually setting of stamps (future or past) breaks that. That means, if you modify stamps manually, your Gets can have indeterminate behavior.
I'd be more than happy to remove the strictness here, and allow anything < Long.MAX_VALUE (if equal, rewrite as now()), with a warning in Put that manually setting of stamp can cause indeterminate behavior, especially in Gets.
In any case, I'd like to see a unit test that verifies the behavior that we do have. There is already a bit of it in o.a.h.h.c.TestClient, but there should be more, perhaps another method or part of testVersions(). It should do insertions with past, present, future stamps, in all different orders, with lots of flushing in between (all issues with stamps / indeterminate behavior are related to multiple storefiles + memstore interaction).
If we can create a new test, with comments explaining what the behavior is, and all prior tests still pass, then I'll be +1 on allowing future stamps.
Stack, Erik, any recollection on why it was left this way?
This should also not have an impact on META where determinate behavior of Gets is quite important.
> [Regression] Cannot save timestamp in the future
> ------------------------------------------------
>
> Key: HBASE-1792
> URL: https://issues.apache.org/jira/browse/HBASE-1792
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Jean-Daniel Cryans
>
> 0.20, compared to previous versions, doesn't let you save with a timestamp in the future and will set it to current time without telling you. This is really bad for users upgrading to 0.20 that were using those timestamps.
> Example:
> hbase(main):004:0> put 'testtable', 'r1', 'f1:c1', 'val', 5373965335336911168
> 0 row(s) in 0.0070 seconds
> hbase(main):005:0> scan 'testtable'
> ROW COLUMN+CELL
> r1 column=f1:c1, timestamp=1251223892010, value=val
> 1 row(s) in 0.0380 seconds
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.