You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "stack (JIRA)" <ji...@apache.org> on 2013/05/22 07:39:22 UTC
[jira] [Resolved] (HBASE-8591) Doc Improvement: Replication blog
[ https://issues.apache.org/jira/browse/HBASE-8591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack resolved HBASE-8591.
--------------------------
Resolution: Fixed
Fix Version/s: 0.98.0
Hadoop Flags: Reviewed
Committed to trunk. Thanks Himanshu (And Chris for review)
> Doc Improvement: Replication blog
> ---------------------------------
>
> Key: HBASE-8591
> URL: https://issues.apache.org/jira/browse/HBASE-8591
> Project: HBase
> Issue Type: Task
> Components: documentation, Replication
> Affects Versions: 0.95.0
> Reporter: Himanshu Vashishtha
> Assignee: Himanshu Vashishtha
> Fix For: 0.98.0
>
> Attachments: HBase-8591.patch, HBase-8591-v1.patch
>
>
> Add a section for source level metrics and some truth about table batch ops at sink.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] [Resolved] (HBASE-8591) Doc Improvement: Replication blog
Posted by Varun Sharma <va...@pinterest.com>.
Hi Himanshu,
I have a follow up question on these metrics. I see that both these metrics
basically diff against the timestamp of the WALEdit. The timestamp on the
WALEdit is generated by the region server in source cluster. So if there is
some clock skew (even 1 second) on the slave cluster, the
ageOfLastAppliedEdit could be somewhat off. On the other hand, I see that
ageOfLastShippedEdit would use the same clock (unless we have region server
failures) for calculating the time on the WALEdit and the time by which it
has been shipped. Also since the shipping is synchronous, we are sure that
the WALEdit has truly been applied on the sink cluster.
Is that correct ?
Varun
On Wed, May 22, 2013 at 11:55 AM, Demai Ni <ni...@gmail.com> wrote:
> Himanshu,
>
> Thanks a lot. Your suggestion saved me a lot of time if have to figure it
> out myself
>
> Demai on the run
>
> On May 22, 2013, at 11:00 AM, Himanshu Vashishtha <hv...@gmail.com>
> wrote:
>
> > Since the source metrics would not be available in case the master
> cluster
> > is down,
> > your best bet would be "sink.ageOfLastAppliedOp" at the slave cluster.
> > Its value is the timestamp of the last WALEdit that is applied to the
> > slave.
> >
> >
> > On Wed, May 22, 2013 at 10:02 AM, N Dm <ni...@gmail.com> wrote:
> >
> >> Stack and Himanshu,
> >>
> >> many thanks for putting this together. Particularly the info from
> >> Replication Metrics, whichs help a lot to understand the replication.
> >>
> >> A quick question on source.ageOfLastShippedOp. In the case that master
> >> cluster down, the user will like to know what the delta between master
> and
> >> slave. In another word, to find out the portion of mutation that already
> >> commit on Master, but haven't replicated yet. Is
> source.ageOfLastShippedOp
> >> the right place to look at? will it be available even with HBASE down?
> >>
> >> Again thanks for the doc and appreciate it.
> >>
> >> Demai
> >>
> >>
> >> On Tue, May 21, 2013 at 10:39 PM, stack (JIRA) <ji...@apache.org> wrote:
> >>
> >>>
> >>> [
> >>>
> >>
> https://issues.apache.org/jira/browse/HBASE-8591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> >> ]
> >>>
> >>> stack resolved HBASE-8591.
> >>> --------------------------
> >>>
> >>> Resolution: Fixed
> >>> Fix Version/s: 0.98.0
> >>> Hadoop Flags: Reviewed
> >>>
> >>> Committed to trunk. Thanks Himanshu (And Chris for review)
> >>>
> >>>> Doc Improvement: Replication blog
> >>>> ---------------------------------
> >>>>
> >>>> Key: HBASE-8591
> >>>> URL: https://issues.apache.org/jira/browse/HBASE-8591
> >>>> Project: HBase
> >>>> Issue Type: Task
> >>>> Components: documentation, Replication
> >>>> Affects Versions: 0.95.0
> >>>> Reporter: Himanshu Vashishtha
> >>>> Assignee: Himanshu Vashishtha
> >>>> Fix For: 0.98.0
> >>>>
> >>>> Attachments: HBase-8591.patch, HBase-8591-v1.patch
> >>>>
> >>>>
> >>>> Add a section for source level metrics and some truth about table
> batch
> >>> ops at sink.
> >>>
> >>> --
> >>> This message is automatically generated by JIRA.
> >>> If you think it was sent incorrectly, please contact your JIRA
> >>> administrators
> >>> For more information on JIRA, see:
> >> http://www.atlassian.com/software/jira
> >>>
> >>
>
Re: [jira] [Resolved] (HBASE-8591) Doc Improvement: Replication blog
Posted by Demai Ni <ni...@gmail.com>.
Himanshu,
Thanks a lot. Your suggestion saved me a lot of time if have to figure it out myself
Demai on the run
On May 22, 2013, at 11:00 AM, Himanshu Vashishtha <hv...@gmail.com> wrote:
> Since the source metrics would not be available in case the master cluster
> is down,
> your best bet would be "sink.ageOfLastAppliedOp" at the slave cluster.
> Its value is the timestamp of the last WALEdit that is applied to the
> slave.
>
>
> On Wed, May 22, 2013 at 10:02 AM, N Dm <ni...@gmail.com> wrote:
>
>> Stack and Himanshu,
>>
>> many thanks for putting this together. Particularly the info from
>> Replication Metrics, whichs help a lot to understand the replication.
>>
>> A quick question on source.ageOfLastShippedOp. In the case that master
>> cluster down, the user will like to know what the delta between master and
>> slave. In another word, to find out the portion of mutation that already
>> commit on Master, but haven't replicated yet. Is source.ageOfLastShippedOp
>> the right place to look at? will it be available even with HBASE down?
>>
>> Again thanks for the doc and appreciate it.
>>
>> Demai
>>
>>
>> On Tue, May 21, 2013 at 10:39 PM, stack (JIRA) <ji...@apache.org> wrote:
>>
>>>
>>> [
>>>
>> https://issues.apache.org/jira/browse/HBASE-8591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
>> ]
>>>
>>> stack resolved HBASE-8591.
>>> --------------------------
>>>
>>> Resolution: Fixed
>>> Fix Version/s: 0.98.0
>>> Hadoop Flags: Reviewed
>>>
>>> Committed to trunk. Thanks Himanshu (And Chris for review)
>>>
>>>> Doc Improvement: Replication blog
>>>> ---------------------------------
>>>>
>>>> Key: HBASE-8591
>>>> URL: https://issues.apache.org/jira/browse/HBASE-8591
>>>> Project: HBase
>>>> Issue Type: Task
>>>> Components: documentation, Replication
>>>> Affects Versions: 0.95.0
>>>> Reporter: Himanshu Vashishtha
>>>> Assignee: Himanshu Vashishtha
>>>> Fix For: 0.98.0
>>>>
>>>> Attachments: HBase-8591.patch, HBase-8591-v1.patch
>>>>
>>>>
>>>> Add a section for source level metrics and some truth about table batch
>>> ops at sink.
>>>
>>> --
>>> This message is automatically generated by JIRA.
>>> If you think it was sent incorrectly, please contact your JIRA
>>> administrators
>>> For more information on JIRA, see:
>> http://www.atlassian.com/software/jira
>>>
>>
Re: [jira] [Resolved] (HBASE-8591) Doc Improvement: Replication blog
Posted by Himanshu Vashishtha <hv...@gmail.com>.
Since the source metrics would not be available in case the master cluster
is down,
your best bet would be "sink.ageOfLastAppliedOp" at the slave cluster.
Its value is the timestamp of the last WALEdit that is applied to the
slave.
On Wed, May 22, 2013 at 10:02 AM, N Dm <ni...@gmail.com> wrote:
> Stack and Himanshu,
>
> many thanks for putting this together. Particularly the info from
> Replication Metrics, whichs help a lot to understand the replication.
>
> A quick question on source.ageOfLastShippedOp. In the case that master
> cluster down, the user will like to know what the delta between master and
> slave. In another word, to find out the portion of mutation that already
> commit on Master, but haven't replicated yet. Is source.ageOfLastShippedOp
> the right place to look at? will it be available even with HBASE down?
>
> Again thanks for the doc and appreciate it.
>
> Demai
>
>
> On Tue, May 21, 2013 at 10:39 PM, stack (JIRA) <ji...@apache.org> wrote:
>
> >
> > [
> >
> https://issues.apache.org/jira/browse/HBASE-8591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> ]
> >
> > stack resolved HBASE-8591.
> > --------------------------
> >
> > Resolution: Fixed
> > Fix Version/s: 0.98.0
> > Hadoop Flags: Reviewed
> >
> > Committed to trunk. Thanks Himanshu (And Chris for review)
> >
> > > Doc Improvement: Replication blog
> > > ---------------------------------
> > >
> > > Key: HBASE-8591
> > > URL: https://issues.apache.org/jira/browse/HBASE-8591
> > > Project: HBase
> > > Issue Type: Task
> > > Components: documentation, Replication
> > > Affects Versions: 0.95.0
> > > Reporter: Himanshu Vashishtha
> > > Assignee: Himanshu Vashishtha
> > > Fix For: 0.98.0
> > >
> > > Attachments: HBase-8591.patch, HBase-8591-v1.patch
> > >
> > >
> > > Add a section for source level metrics and some truth about table batch
> > ops at sink.
> >
> > --
> > This message is automatically generated by JIRA.
> > If you think it was sent incorrectly, please contact your JIRA
> > administrators
> > For more information on JIRA, see:
> http://www.atlassian.com/software/jira
> >
>
Re: [jira] [Resolved] (HBASE-8591) Doc Improvement: Replication blog
Posted by N Dm <ni...@gmail.com>.
Stack and Himanshu,
many thanks for putting this together. Particularly the info from
Replication Metrics, whichs help a lot to understand the replication.
A quick question on source.ageOfLastShippedOp. In the case that master
cluster down, the user will like to know what the delta between master and
slave. In another word, to find out the portion of mutation that already
commit on Master, but haven't replicated yet. Is source.ageOfLastShippedOp
the right place to look at? will it be available even with HBASE down?
Again thanks for the doc and appreciate it.
Demai
On Tue, May 21, 2013 at 10:39 PM, stack (JIRA) <ji...@apache.org> wrote:
>
> [
> https://issues.apache.org/jira/browse/HBASE-8591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel]
>
> stack resolved HBASE-8591.
> --------------------------
>
> Resolution: Fixed
> Fix Version/s: 0.98.0
> Hadoop Flags: Reviewed
>
> Committed to trunk. Thanks Himanshu (And Chris for review)
>
> > Doc Improvement: Replication blog
> > ---------------------------------
> >
> > Key: HBASE-8591
> > URL: https://issues.apache.org/jira/browse/HBASE-8591
> > Project: HBase
> > Issue Type: Task
> > Components: documentation, Replication
> > Affects Versions: 0.95.0
> > Reporter: Himanshu Vashishtha
> > Assignee: Himanshu Vashishtha
> > Fix For: 0.98.0
> >
> > Attachments: HBase-8591.patch, HBase-8591-v1.patch
> >
> >
> > Add a section for source level metrics and some truth about table batch
> ops at sink.
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA
> administrators
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>