You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by David Quigley <dq...@gmail.com> on 2014/04/08 16:19:54 UTC
SnapshotInputFormat long initialization time.
Hi all,
I backported SnapshotInputFormat to hbase 0.94. This resulted in a huge
performance gain while the scan is running. Here is the corresponding jira
https://issues.apache.org/jira/browse/HBASE-8369. The overall performance
gain is mitigated by a very long initialization period. It can take up to
30 minutes after the snapshot completes for the MR job to actually begin.
Does anybody have experience running SnapshotInputFormay on 0.94 or 0.98?
Are you seeing the same long initialization period?
Thanks!
Dave
Re: SnapshotInputFormat long initialization time.
Posted by David Quigley <dq...@gmail.com>.
No haven't seen that, I think I backported just before that jira was
created. Thanks! ill check it out
On Tue, Apr 8, 2014 at 7:45 AM, Ted Yu <yu...@gmail.com> wrote:
> Have you looked at HBASE-10642 which was integrated to 0.94.18 ?
>
> Cheers
>
>
> On Tue, Apr 8, 2014 at 8:19 AM, David Quigley <dq...@gmail.com>wrote:
>
>> Hi all,
>>
>> I backported SnapshotInputFormat to hbase 0.94. This resulted in a huge
>> performance gain while the scan is running. Here is the corresponding jira
>> https://issues.apache.org/jira/browse/HBASE-8369. The overall
>> performance gain is mitigated by a very long initialization period. It can
>> take up to 30 minutes after the snapshot completes for the MR job to
>> actually begin.
>>
>> Does anybody have experience running SnapshotInputFormay on 0.94 or 0.98?
>> Are you seeing the same long initialization period?
>>
>> Thanks!
>> Dave
>>
>
>
Re: SnapshotInputFormat long initialization time.
Posted by David Quigley <dq...@gmail.com>.
No haven't seen that, I think I backported just before that jira was
created. Thanks! ill check it out
On Tue, Apr 8, 2014 at 7:45 AM, Ted Yu <yu...@gmail.com> wrote:
> Have you looked at HBASE-10642 which was integrated to 0.94.18 ?
>
> Cheers
>
>
> On Tue, Apr 8, 2014 at 8:19 AM, David Quigley <dq...@gmail.com>wrote:
>
>> Hi all,
>>
>> I backported SnapshotInputFormat to hbase 0.94. This resulted in a huge
>> performance gain while the scan is running. Here is the corresponding jira
>> https://issues.apache.org/jira/browse/HBASE-8369. The overall
>> performance gain is mitigated by a very long initialization period. It can
>> take up to 30 minutes after the snapshot completes for the MR job to
>> actually begin.
>>
>> Does anybody have experience running SnapshotInputFormay on 0.94 or 0.98?
>> Are you seeing the same long initialization period?
>>
>> Thanks!
>> Dave
>>
>
>
Re: SnapshotInputFormat long initialization time.
Posted by David Quigley <dq...@gmail.com>.
No haven't seen that, I think I backported just before that jira was
created. Thanks! ill check it out
On Tue, Apr 8, 2014 at 7:45 AM, Ted Yu <yu...@gmail.com> wrote:
> Have you looked at HBASE-10642 which was integrated to 0.94.18 ?
>
> Cheers
>
>
> On Tue, Apr 8, 2014 at 8:19 AM, David Quigley <dq...@gmail.com>wrote:
>
>> Hi all,
>>
>> I backported SnapshotInputFormat to hbase 0.94. This resulted in a huge
>> performance gain while the scan is running. Here is the corresponding jira
>> https://issues.apache.org/jira/browse/HBASE-8369. The overall
>> performance gain is mitigated by a very long initialization period. It can
>> take up to 30 minutes after the snapshot completes for the MR job to
>> actually begin.
>>
>> Does anybody have experience running SnapshotInputFormay on 0.94 or 0.98?
>> Are you seeing the same long initialization period?
>>
>> Thanks!
>> Dave
>>
>
>
Re: SnapshotInputFormat long initialization time.
Posted by David Quigley <dq...@gmail.com>.
No haven't seen that, I think I backported just before that jira was
created. Thanks! ill check it out
On Tue, Apr 8, 2014 at 7:45 AM, Ted Yu <yu...@gmail.com> wrote:
> Have you looked at HBASE-10642 which was integrated to 0.94.18 ?
>
> Cheers
>
>
> On Tue, Apr 8, 2014 at 8:19 AM, David Quigley <dq...@gmail.com>wrote:
>
>> Hi all,
>>
>> I backported SnapshotInputFormat to hbase 0.94. This resulted in a huge
>> performance gain while the scan is running. Here is the corresponding jira
>> https://issues.apache.org/jira/browse/HBASE-8369. The overall
>> performance gain is mitigated by a very long initialization period. It can
>> take up to 30 minutes after the snapshot completes for the MR job to
>> actually begin.
>>
>> Does anybody have experience running SnapshotInputFormay on 0.94 or 0.98?
>> Are you seeing the same long initialization period?
>>
>> Thanks!
>> Dave
>>
>
>
Re: SnapshotInputFormat long initialization time.
Posted by Ted Yu <yu...@gmail.com>.
Have you looked at HBASE-10642 which was integrated to 0.94.18 ?
Cheers
On Tue, Apr 8, 2014 at 8:19 AM, David Quigley <dq...@gmail.com> wrote:
> Hi all,
>
> I backported SnapshotInputFormat to hbase 0.94. This resulted in a huge
> performance gain while the scan is running. Here is the corresponding jira
> https://issues.apache.org/jira/browse/HBASE-8369. The overall performance
> gain is mitigated by a very long initialization period. It can take up to
> 30 minutes after the snapshot completes for the MR job to actually begin.
>
> Does anybody have experience running SnapshotInputFormay on 0.94 or 0.98?
> Are you seeing the same long initialization period?
>
> Thanks!
> Dave
>
Re: SnapshotInputFormat long initialization time.
Posted by Ted Yu <yu...@gmail.com>.
Have you looked at HBASE-10642 which was integrated to 0.94.18 ?
Cheers
On Tue, Apr 8, 2014 at 8:19 AM, David Quigley <dq...@gmail.com> wrote:
> Hi all,
>
> I backported SnapshotInputFormat to hbase 0.94. This resulted in a huge
> performance gain while the scan is running. Here is the corresponding jira
> https://issues.apache.org/jira/browse/HBASE-8369. The overall performance
> gain is mitigated by a very long initialization period. It can take up to
> 30 minutes after the snapshot completes for the MR job to actually begin.
>
> Does anybody have experience running SnapshotInputFormay on 0.94 or 0.98?
> Are you seeing the same long initialization period?
>
> Thanks!
> Dave
>
Re: SnapshotInputFormat long initialization time.
Posted by Ted Yu <yu...@gmail.com>.
Have you looked at HBASE-10642 which was integrated to 0.94.18 ?
Cheers
On Tue, Apr 8, 2014 at 8:19 AM, David Quigley <dq...@gmail.com> wrote:
> Hi all,
>
> I backported SnapshotInputFormat to hbase 0.94. This resulted in a huge
> performance gain while the scan is running. Here is the corresponding jira
> https://issues.apache.org/jira/browse/HBASE-8369. The overall performance
> gain is mitigated by a very long initialization period. It can take up to
> 30 minutes after the snapshot completes for the MR job to actually begin.
>
> Does anybody have experience running SnapshotInputFormay on 0.94 or 0.98?
> Are you seeing the same long initialization period?
>
> Thanks!
> Dave
>
Re: SnapshotInputFormat long initialization time.
Posted by Ted Yu <yu...@gmail.com>.
Have you looked at HBASE-10642 which was integrated to 0.94.18 ?
Cheers
On Tue, Apr 8, 2014 at 8:19 AM, David Quigley <dq...@gmail.com> wrote:
> Hi all,
>
> I backported SnapshotInputFormat to hbase 0.94. This resulted in a huge
> performance gain while the scan is running. Here is the corresponding jira
> https://issues.apache.org/jira/browse/HBASE-8369. The overall performance
> gain is mitigated by a very long initialization period. It can take up to
> 30 minutes after the snapshot completes for the MR job to actually begin.
>
> Does anybody have experience running SnapshotInputFormay on 0.94 or 0.98?
> Are you seeing the same long initialization period?
>
> Thanks!
> Dave
>