You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "stack (JIRA)" <ji...@apache.org> on 2008/06/26 05:10:45 UTC
[jira] Created: (HBASE-711) Complain if clock skew across the
cluster is badly out of sync
Complain if clock skew across the cluster is badly out of sync
--------------------------------------------------------------
Key: HBASE-711
URL: https://issues.apache.org/jira/browse/HBASE-711
Project: Hadoop HBase
Issue Type: New Feature
Reporter: stack
Priority: Minor
hbase-710 and hbase-609 are issues where the system has broken in presence of clock skew over the cluster. Would be a nice service if master could flag very bad clock skew. Regionservers could report their local time when they ping the master. It could do a compare.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-711) Complain if clock skew across the
cluster is badly out of sync
Posted by "Jim Kellerman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jim Kellerman updated HBASE-711:
--------------------------------
Fix Version/s: 0.19.0
> Complain if clock skew across the cluster is badly out of sync
> --------------------------------------------------------------
>
> Key: HBASE-711
> URL: https://issues.apache.org/jira/browse/HBASE-711
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: stack
> Assignee: Jim Kellerman
> Priority: Minor
> Fix For: 0.19.0
>
>
> hbase-710 and hbase-609 are issues where the system has broken in presence of clock skew over the cluster. Would be a nice service if master could flag very bad clock skew. Regionservers could report their local time when they ping the master. It could do a compare.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-711) Complain if clock skew across the
cluster is badly out of sync
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12620397#action_12620397 ]
stack commented on HBASE-711:
-----------------------------
+1 on the latter suggestion regards HLog names.
On the renaming of regions to include start/end keys, when region splits, its end-key will be less than that of its parents so daughter will go in ahead of its parent in the meta. Will this be a problem? Will it break our getRowAtOrBefore?
> Complain if clock skew across the cluster is badly out of sync
> --------------------------------------------------------------
>
> Key: HBASE-711
> URL: https://issues.apache.org/jira/browse/HBASE-711
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: stack
> Assignee: Jim Kellerman
> Priority: Minor
>
> hbase-710 and hbase-609 are issues where the system has broken in presence of clock skew over the cluster. Would be a nice service if master could flag very bad clock skew. Regionservers could report their local time when they ping the master. It could do a compare.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-711) Complain if clock skew across the
cluster is badly out of sync
Posted by "Jim Kellerman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jim Kellerman updated HBASE-711:
--------------------------------
Priority: Blocker (was: Minor)
> Complain if clock skew across the cluster is badly out of sync
> --------------------------------------------------------------
>
> Key: HBASE-711
> URL: https://issues.apache.org/jira/browse/HBASE-711
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: Jim Kellerman
> Priority: Blocker
> Fix For: 0.19.0
>
>
> hbase-710 and hbase-609 are issues where the system has broken in presence of clock skew over the cluster. Would be a nice service if master could flag very bad clock skew. Regionservers could report their local time when they ping the master. It could do a compare.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-711) Complain if clock skew across the
cluster is badly out of sync
Posted by "Jim Kellerman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12622730#action_12622730 ]
Jim Kellerman commented on HBASE-711:
-------------------------------------
If we require all the servers to run nntpd (or equivalent) then this should largely be a non-issue with respect to either region names or HLog names.
The only thing we might want to do is have the master examine the HServerInfo passed from the region server in the regionServerStartup message and if HServerInfo.getStartCode() (which is the time the region server started) is larger than some delta from the master's current clock time, the master should throw an exception that would cause the region server to shut down.
> Complain if clock skew across the cluster is badly out of sync
> --------------------------------------------------------------
>
> Key: HBASE-711
> URL: https://issues.apache.org/jira/browse/HBASE-711
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: stack
> Assignee: Jim Kellerman
> Priority: Minor
>
> hbase-710 and hbase-609 are issues where the system has broken in presence of clock skew over the cluster. Would be a nice service if master could flag very bad clock skew. Regionservers could report their local time when they ping the master. It could do a compare.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-711) Complain if clock skew across the
cluster is badly out of sync
Posted by "Jim Kellerman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jim Kellerman updated HBASE-711:
--------------------------------
Issue Type: Sub-task (was: New Feature)
Parent: HBASE-678
> Complain if clock skew across the cluster is badly out of sync
> --------------------------------------------------------------
>
> Key: HBASE-711
> URL: https://issues.apache.org/jira/browse/HBASE-711
> Project: Hadoop HBase
> Issue Type: Sub-task
> Reporter: stack
> Assignee: Jim Kellerman
> Priority: Minor
> Fix For: 0.19.0
>
>
> hbase-710 and hbase-609 are issues where the system has broken in presence of clock skew over the cluster. Would be a nice service if master could flag very bad clock skew. Regionservers could report their local time when they ping the master. It could do a compare.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-711) Complain if clock skew across the
cluster is badly out of sync
Posted by "Jim Kellerman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12620392#action_12620392 ]
Jim Kellerman commented on HBASE-711:
-------------------------------------
Another place we use the current time is in the creation of HLog files.
Proposal: instead of using the current time, use the current sequence number (with leading zero padding out to some large size so the files still sort properly). The files would still be increasing, and sort properly.
> Complain if clock skew across the cluster is badly out of sync
> --------------------------------------------------------------
>
> Key: HBASE-711
> URL: https://issues.apache.org/jira/browse/HBASE-711
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: stack
> Assignee: Jim Kellerman
> Priority: Minor
>
> hbase-710 and hbase-609 are issues where the system has broken in presence of clock skew over the cluster. Would be a nice service if master could flag very bad clock skew. Regionservers could report their local time when they ping the master. It could do a compare.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-711) Complain if clock skew across the
cluster is badly out of sync
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12615030#action_12615030 ]
Jean-Daniel Cryans commented on HBASE-711:
------------------------------------------
+1
> Complain if clock skew across the cluster is badly out of sync
> --------------------------------------------------------------
>
> Key: HBASE-711
> URL: https://issues.apache.org/jira/browse/HBASE-711
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: stack
> Assignee: Jim Kellerman
> Priority: Minor
>
> hbase-710 and hbase-609 are issues where the system has broken in presence of clock skew over the cluster. Would be a nice service if master could flag very bad clock skew. Regionservers could report their local time when they ping the master. It could do a compare.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-711) Complain if clock skew across the
cluster is badly out of sync
Posted by "Jim Kellerman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jim Kellerman updated HBASE-711:
--------------------------------
Issue Type: Bug (was: Sub-task)
Parent: (was: HBASE-678)
> Complain if clock skew across the cluster is badly out of sync
> --------------------------------------------------------------
>
> Key: HBASE-711
> URL: https://issues.apache.org/jira/browse/HBASE-711
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: Jim Kellerman
> Priority: Minor
> Fix For: 0.19.0
>
>
> hbase-710 and hbase-609 are issues where the system has broken in presence of clock skew over the cluster. Would be a nice service if master could flag very bad clock skew. Regionservers could report their local time when they ping the master. It could do a compare.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-711) Complain if clock skew across the
cluster is badly out of sync
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12615052#action_12615052 ]
stack commented on HBASE-711:
-----------------------------
Added NTP into the requirements section of the 'Gettting Started' documentation.
> Complain if clock skew across the cluster is badly out of sync
> --------------------------------------------------------------
>
> Key: HBASE-711
> URL: https://issues.apache.org/jira/browse/HBASE-711
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: stack
> Assignee: Jim Kellerman
> Priority: Minor
>
> hbase-710 and hbase-609 are issues where the system has broken in presence of clock skew over the cluster. Would be a nice service if master could flag very bad clock skew. Regionservers could report their local time when they ping the master. It could do a compare.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-711) Complain if clock skew across the
cluster is badly out of sync
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12622943#action_12622943 ]
stack commented on HBASE-711:
-----------------------------
That'd be useful. Maybe a warning is good enough on master side rather than a shutdown of regionserver since we *think* we're immune to clock skew at this stage.
> Complain if clock skew across the cluster is badly out of sync
> --------------------------------------------------------------
>
> Key: HBASE-711
> URL: https://issues.apache.org/jira/browse/HBASE-711
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: stack
> Assignee: Jim Kellerman
> Priority: Minor
>
> hbase-710 and hbase-609 are issues where the system has broken in presence of clock skew over the cluster. Would be a nice service if master could flag very bad clock skew. Regionservers could report their local time when they ping the master. It could do a compare.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HBASE-711) Complain if clock skew across the
cluster is badly out of sync
Posted by "Jim Kellerman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jim Kellerman reassigned HBASE-711:
-----------------------------------
Assignee: Jim Kellerman
> Complain if clock skew across the cluster is badly out of sync
> --------------------------------------------------------------
>
> Key: HBASE-711
> URL: https://issues.apache.org/jira/browse/HBASE-711
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: stack
> Assignee: Jim Kellerman
> Priority: Minor
>
> hbase-710 and hbase-609 are issues where the system has broken in presence of clock skew over the cluster. Would be a nice service if master could flag very bad clock skew. Regionservers could report their local time when they ping the master. It could do a compare.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-711) Complain if clock skew across the
cluster is badly out of sync
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12620406#action_12620406 ]
stack commented on HBASE-711:
-----------------------------
IIUC, region A with key a start key of 10 and an end key of 12 will be named A,10,12. When it splits, the lower halves new name will be A,10,11 which sorts before A,10,12. We don't want that, right?
> Complain if clock skew across the cluster is badly out of sync
> --------------------------------------------------------------
>
> Key: HBASE-711
> URL: https://issues.apache.org/jira/browse/HBASE-711
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: stack
> Assignee: Jim Kellerman
> Priority: Minor
>
> hbase-710 and hbase-609 are issues where the system has broken in presence of clock skew over the cluster. Would be a nice service if master could flag very bad clock skew. Regionservers could report their local time when they ping the master. It could do a compare.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-711) Complain if clock skew across the
cluster is badly out of sync
Posted by "Jim Kellerman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12615029#action_12615029 ]
Jim Kellerman commented on HBASE-711:
-------------------------------------
We need to document in the FAQ or somewhere, that servers should be running nntpd (or the equivalent on other platforms)
> Complain if clock skew across the cluster is badly out of sync
> --------------------------------------------------------------
>
> Key: HBASE-711
> URL: https://issues.apache.org/jira/browse/HBASE-711
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: stack
> Assignee: Jim Kellerman
> Priority: Minor
>
> hbase-710 and hbase-609 are issues where the system has broken in presence of clock skew over the cluster. Would be a nice service if master could flag very bad clock skew. Regionservers could report their local time when they ping the master. It could do a compare.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-711) Complain if clock skew across the
cluster is badly out of sync
Posted by "Jim Kellerman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12620080#action_12620080 ]
Jim Kellerman commented on HBASE-711:
-------------------------------------
Noting the clock skew won't work because HRegionInfo constructor uses clock time to set the region name.
However, do we really need to disambiguate region names with the clock time?
Since region name = table name + start key + end key, can it ever be the case that you would have two regions with the same name?
> Complain if clock skew across the cluster is badly out of sync
> --------------------------------------------------------------
>
> Key: HBASE-711
> URL: https://issues.apache.org/jira/browse/HBASE-711
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: stack
> Assignee: Jim Kellerman
> Priority: Minor
>
> hbase-710 and hbase-609 are issues where the system has broken in presence of clock skew over the cluster. Would be a nice service if master could flag very bad clock skew. Regionservers could report their local time when they ping the master. It could do a compare.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-711) Complain if clock skew across the
cluster is badly out of sync
Posted by "Jim Kellerman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12620402#action_12620402 ]
Jim Kellerman commented on HBASE-711:
-------------------------------------
HRegion plays around with current time to create region names for split children. If we named regions as I have suggested above, this will no longer be necessary.
All the other references to System.currentTimeMillis() are used for interval computations and so it doesn't matter if the clock is skewed or not.
> Complain if clock skew across the cluster is badly out of sync
> --------------------------------------------------------------
>
> Key: HBASE-711
> URL: https://issues.apache.org/jira/browse/HBASE-711
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: stack
> Assignee: Jim Kellerman
> Priority: Minor
>
> hbase-710 and hbase-609 are issues where the system has broken in presence of clock skew over the cluster. Would be a nice service if master could flag very bad clock skew. Regionservers could report their local time when they ping the master. It could do a compare.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-711) Complain if clock skew across the
cluster is badly out of sync
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12622935#action_12622935 ]
stack commented on HBASE-711:
-----------------------------
FYI, in case you hadn't seen it, we added this to split code a while back:
{code}
// Calculate regionid to use. Can't be less than that of parent else
// it'll insert into wrong location over in .META. table: HBASE-710.
long rid = System.currentTimeMillis();
if (rid < this.regionInfo.getRegionId()) {
LOG.warn("Clock skew; parent regions id is " +
this.regionInfo.getRegionId() + " but current time here is " + rid);
rid = this.regionInfo.getRegionId() + 1;
}
{code}
Makes it so splits always sort after parents whatever the clockskew.
> Complain if clock skew across the cluster is badly out of sync
> --------------------------------------------------------------
>
> Key: HBASE-711
> URL: https://issues.apache.org/jira/browse/HBASE-711
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: stack
> Assignee: Jim Kellerman
> Priority: Minor
>
> hbase-710 and hbase-609 are issues where the system has broken in presence of clock skew over the cluster. Would be a nice service if master could flag very bad clock skew. Regionservers could report their local time when they ping the master. It could do a compare.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-711) Complain if clock skew across the
cluster is badly out of sync
Posted by "Jim Kellerman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12620099#action_12620099 ]
Jim Kellerman commented on HBASE-711:
-------------------------------------
I was incorrect in previous comment. Currently, region name = table name + start key + region id.
Proposal: make region name = table name + start key + end key. This would eliminate the need for the region id.
> Complain if clock skew across the cluster is badly out of sync
> --------------------------------------------------------------
>
> Key: HBASE-711
> URL: https://issues.apache.org/jira/browse/HBASE-711
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: stack
> Assignee: Jim Kellerman
> Priority: Minor
>
> hbase-710 and hbase-609 are issues where the system has broken in presence of clock skew over the cluster. Would be a nice service if master could flag very bad clock skew. Regionservers could report their local time when they ping the master. It could do a compare.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-711) Complain if clock skew across the
cluster is badly out of sync
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12608501#action_12608501 ]
stack commented on HBASE-711:
-----------------------------
HBASE-710 now emits on regionserver a warning at split time if difference between parent and daughter regionid/timestamp is such that current time on regionserver is less than parent regionid/timestamp. This is not sufficient to satisfy this issue but its a start.
> Complain if clock skew across the cluster is badly out of sync
> --------------------------------------------------------------
>
> Key: HBASE-711
> URL: https://issues.apache.org/jira/browse/HBASE-711
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: stack
> Priority: Minor
>
> hbase-710 and hbase-609 are issues where the system has broken in presence of clock skew over the cluster. Would be a nice service if master could flag very bad clock skew. Regionservers could report their local time when they ping the master. It could do a compare.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-711) Complain if clock skew across the
cluster is badly out of sync
Posted by "Jim Kellerman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12622940#action_12622940 ]
Jim Kellerman commented on HBASE-711:
-------------------------------------
Yes, saw that. Coupled with the use of nntpd, this is largely a non-issue. However I thought we could add something to the startup sequence in which:
> The master examines the HServerInfo passed from the region server in the regionServerStartup message
> and if HServerInfo.getStartCode() (which is the time the region server started) is larger than some delta
> from the master's current clock time, the master should throw an exception that would cause the region
> server to shut down.
Think that is necessary? Or should we close this issue as either wontfix or invalid?
> Complain if clock skew across the cluster is badly out of sync
> --------------------------------------------------------------
>
> Key: HBASE-711
> URL: https://issues.apache.org/jira/browse/HBASE-711
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: stack
> Assignee: Jim Kellerman
> Priority: Minor
>
> hbase-710 and hbase-609 are issues where the system has broken in presence of clock skew over the cluster. Would be a nice service if master could flag very bad clock skew. Regionservers could report their local time when they ping the master. It could do a compare.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HBASE-711) Complain if clock skew across the
cluster is badly out of sync
Posted by "Jim Kellerman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jim Kellerman reassigned HBASE-711:
-----------------------------------
Assignee: Jim Kellerman
> Complain if clock skew across the cluster is badly out of sync
> --------------------------------------------------------------
>
> Key: HBASE-711
> URL: https://issues.apache.org/jira/browse/HBASE-711
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: stack
> Assignee: Jim Kellerman
> Priority: Minor
>
> hbase-710 and hbase-609 are issues where the system has broken in presence of clock skew over the cluster. Would be a nice service if master could flag very bad clock skew. Regionservers could report their local time when they ping the master. It could do a compare.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-711) Complain if clock skew across the
cluster is badly out of sync
Posted by "Billy Pearson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12614947#action_12614947 ]
Billy Pearson commented on HBASE-711:
-------------------------------------
How about we stored a variable for the skew for the region server?
Example when the region server checks in with the the master on startup and the master sends back its time and the region server stores the skew form its clock to the master clock?
We would need to factor this skew in to the internal code that call for timestamp not sure how much work that would be but an idea.
> Complain if clock skew across the cluster is badly out of sync
> --------------------------------------------------------------
>
> Key: HBASE-711
> URL: https://issues.apache.org/jira/browse/HBASE-711
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: stack
> Assignee: Jim Kellerman
> Priority: Minor
>
> hbase-710 and hbase-609 are issues where the system has broken in presence of clock skew over the cluster. Would be a nice service if master could flag very bad clock skew. Regionservers could report their local time when they ping the master. It could do a compare.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-711) Complain if clock skew across the
cluster is badly out of sync
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-711:
------------------------
Fix Version/s: (was: 0.20.0)
Moving out of 0.20.0. Non-critical. If a patch shows up before release, we'll add it.
> Complain if clock skew across the cluster is badly out of sync
> --------------------------------------------------------------
>
> Key: HBASE-711
> URL: https://issues.apache.org/jira/browse/HBASE-711
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: Jim Kellerman
> Priority: Minor
>
> hbase-710 and hbase-609 are issues where the system has broken in presence of clock skew over the cluster. Would be a nice service if master could flag very bad clock skew. Regionservers could report their local time when they ping the master. It could do a compare.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-711) Complain if clock skew across the
cluster is badly out of sync
Posted by "Jim Kellerman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12620454#action_12620454 ]
Jim Kellerman commented on HBASE-711:
-------------------------------------
> stack - 06/Aug/08 12:57 PM
> IIUC, region A with key a start key of 10 and an end key of 12 will be named A,10,12. When it splits, the lower
> halves new name will be A,10,11 which sorts before A,10,12. We don't want that, right?
Yes, this is a problem given that we use getRowAtOrBefore. If we exposed something like getRowAtOrAfter, this would work.. I have been playing around with strategies that do not require additional APIs, but have not been successful in constructing a key that would sort to the right spot if we got a result back that said that the region was split,offline. I think it is do-able, but just don't have an exact handle on it yet.
> Complain if clock skew across the cluster is badly out of sync
> --------------------------------------------------------------
>
> Key: HBASE-711
> URL: https://issues.apache.org/jira/browse/HBASE-711
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: stack
> Assignee: Jim Kellerman
> Priority: Minor
>
> hbase-710 and hbase-609 are issues where the system has broken in presence of clock skew over the cluster. Would be a nice service if master could flag very bad clock skew. Regionservers could report their local time when they ping the master. It could do a compare.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-711) Complain if clock skew across the
cluster is badly out of sync
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-711:
------------------------
Fix Version/s: (was: 0.19.0)
0.20.0
Priority: Minor (was: Blocker)
Moved out of 0.19.0.
> Complain if clock skew across the cluster is badly out of sync
> --------------------------------------------------------------
>
> Key: HBASE-711
> URL: https://issues.apache.org/jira/browse/HBASE-711
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: Jim Kellerman
> Priority: Minor
> Fix For: 0.20.0
>
>
> hbase-710 and hbase-609 are issues where the system has broken in presence of clock skew over the cluster. Would be a nice service if master could flag very bad clock skew. Regionservers could report their local time when they ping the master. It could do a compare.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-711) Complain if clock skew across the
cluster is badly out of sync
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12615031#action_12615031 ]
Andrew Purtell commented on HBASE-711:
--------------------------------------
+1 Definitely not an inconvenience to a professional to require it. Should get prominent mention. Should be one of the first things any competent administrator of a distributed system sets up anyhow. (Assuming I am an example of such a thing :-) )
> Complain if clock skew across the cluster is badly out of sync
> --------------------------------------------------------------
>
> Key: HBASE-711
> URL: https://issues.apache.org/jira/browse/HBASE-711
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: stack
> Assignee: Jim Kellerman
> Priority: Minor
>
> hbase-710 and hbase-609 are issues where the system has broken in presence of clock skew over the cluster. Would be a nice service if master could flag very bad clock skew. Regionservers could report their local time when they ping the master. It could do a compare.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HBASE-711) Complain if clock skew across the
cluster is badly out of sync
Posted by "Jim Kellerman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jim Kellerman reassigned HBASE-711:
-----------------------------------
Assignee: (was: Jim Kellerman)
> Complain if clock skew across the cluster is badly out of sync
> --------------------------------------------------------------
>
> Key: HBASE-711
> URL: https://issues.apache.org/jira/browse/HBASE-711
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: stack
> Priority: Minor
>
> hbase-710 and hbase-609 are issues where the system has broken in presence of clock skew over the cluster. Would be a nice service if master could flag very bad clock skew. Regionservers could report their local time when they ping the master. It could do a compare.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.