You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by Suresh Srinivas <su...@hortonworks.com> on 2013/05/01 20:54:30 UTC
[VOTE] Merge HDFS-2802 snapshot feature to trunk
This is a follow up to my earlier heads up about merging Snapshot feature
to trunk - http://markmail.org/message/ixkyku2cebkewnzy. I am happy to
announce that we have completed the development of the feature. It is ready
to be merged into trunk.
Development of snapshot feature is tracked in the jira
https://issues.apache.org/jira/browse/HDFS-2802. This is an important
feature for HDFS. Please see a brief presentation that describes the
feature at a highlevel from the Snapshot discussion meetup we had a while
back -
https://issues.apache.org/jira/secure/attachment/12552861/Snapshots.pdf.
Details of development and testing:
Development has been done in a separate branch -
https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2802. The
updated design is posted at -
https://issues.apache.org/jira/secure/attachment/12581376/Snapshots20120429.pdf.
The feature development has involved close to 120 subtasks and close to 25K
lines of code.
A lot of unit tests have been added as a part of the feature. We also have
been testing this in a cluster of 5 nodes with a long running test that
mimics a real cluster usage with emphasis on use cases related to snapshot.
We are also testing non-snapshot code path running tests on a separate
cluster without configuring snapshot. Please see the test plan
https://issues.apache.org/jira/secure/attachment/12575442/snapshot-testplan.pdf
for
the details. Once the feature is merged into trunk, we will continue to
test and fix any bugs that may be found on the trunk.
This feature is a result of many people's collaboration. The bulk of the
code and design work was done by Nicholas Sze, Jing Zhao, Hari Mankude,
Brandon Li, Aprit Agarwal, Sanjay Radia and me. Thanks to Ramya Sunil for
doing feature testing and finding many bugs. Aaron Myers, Konstantin
Shvachko, Allen Wittenauer, Chris Nauroth, Todd Lipcon, Michael Stack, Eli
Collins, Lars Hofhansl
and many others contributed to the feature definition, design and code
reviews on the jiras and during meetup.
This vote runs for a week and closes on 5/8/2013 at 12:00 pm. Here is my +1
for the merge.
Regards,
Suresh
Re: [VOTE] Merge HDFS-2802 snapshot feature to trunk
Posted by Suresh Srinivas <su...@hortonworks.com>.
HDFS-2802 branch has been merged into trunk. Thanks Nicholas for
maintaining the merge patch and merging the branch into trunk.
Regards,
Suresh
On Wed, May 8, 2013 at 1:16 PM, Suresh Srinivas <su...@hortonworks.com>wrote:
> With 3 +1 binding votes, the vote to merge Snapshot feature from HDFS-2802
> branch to trunk passes. The code will be merged soon into. The remaining
> open issues related to this feature will be addressed in trunk.
>
> Thanks to everyone who voted.
>
> Regards,
> Suresh
>
>
> On Wed, May 8, 2013 at 1:12 PM, Suresh Srinivas <su...@hortonworks.com>wrote:
>
>> Aaron, I have created HDFS-4808 to discuss the API issues you brought up.
>> I agree that this needs to be resolved before merging the code to branch-2.
>> Thanks for voting.
>>
>>
>> On Tue, May 7, 2013 at 5:29 PM, Aaron T. Myers <at...@cloudera.com> wrote:
>>
>>> I'm +1 as well on merging this branch to trunk. I've thoroughly
>>> reviewed/discussed the design and have personally reviewed much of the
>>> code
>>> as well. Good stuff.
>>>
>>> I would have preferred to wrap up the discussion regarding which API
>>> should
>>> include the snapshot create/delete operations before the merge, but I
>>> don't
>>> think that should hold up the merge to trunk. No reason we can't discuss
>>> it
>>> afterward, though I'd really like to come to a conclusion about that
>>> before
>>> we merge it to branch-2. Suresh, I believe the last action on that thread
>>> was you asking Nicholas to move the discussion to a JIRA, but I don't
>>> think
>>> that JIRA ever got filed. Mind doing that?
>>>
>>>
>>> --
>>> Aaron T. Myers
>>> Software Engineer, Cloudera
>>>
>>>
>>> On Wed, May 1, 2013 at 11:54 AM, Suresh Srinivas <suresh@hortonworks.com
>>> >wrote:
>>>
>>> > This is a follow up to my earlier heads up about merging Snapshot
>>> feature
>>> > to trunk - http://markmail.org/message/ixkyku2cebkewnzy. I am happy to
>>> > announce that we have completed the development of the feature. It is
>>> ready
>>> > to be merged into trunk.
>>> >
>>> > Development of snapshot feature is tracked in the jira
>>> > https://issues.apache.org/jira/browse/HDFS-2802. This is an important
>>> > feature for HDFS. Please see a brief presentation that describes the
>>> > feature at a highlevel from the Snapshot discussion meetup we had a
>>> while
>>> > back -
>>> >
>>> https://issues.apache.org/jira/secure/attachment/12552861/Snapshots.pdf.
>>> >
>>> >
>>> > Details of development and testing:
>>> > Development has been done in a separate branch -
>>> > https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2802. The
>>> > updated design is posted at -
>>> >
>>> >
>>> https://issues.apache.org/jira/secure/attachment/12581376/Snapshots20120429.pdf
>>> > .
>>> > The feature development has involved close to 120 subtasks and close
>>> to 25K
>>> > lines of code.
>>> >
>>> > A lot of unit tests have been added as a part of the feature. We also
>>> have
>>> > been testing this in a cluster of 5 nodes with a long running test that
>>> > mimics a real cluster usage with emphasis on use cases related to
>>> snapshot.
>>> > We are also testing non-snapshot code path running tests on a separate
>>> > cluster without configuring snapshot. Please see the test plan
>>> >
>>> >
>>> https://issues.apache.org/jira/secure/attachment/12575442/snapshot-testplan.pdf
>>> > for
>>> > the details. Once the feature is merged into trunk, we will continue to
>>> > test and fix any bugs that may be found on the trunk.
>>> >
>>> > This feature is a result of many people's collaboration. The bulk of
>>> the
>>> > code and design work was done by Nicholas Sze, Jing Zhao, Hari Mankude,
>>> > Brandon Li, Aprit Agarwal, Sanjay Radia and me. Thanks to Ramya Sunil
>>> for
>>> > doing feature testing and finding many bugs. Aaron Myers, Konstantin
>>> > Shvachko, Allen Wittenauer, Chris Nauroth, Todd Lipcon, Michael Stack,
>>> Eli
>>> > Collins, Lars Hofhansl
>>> > and many others contributed to the feature definition, design and code
>>> > reviews on the jiras and during meetup.
>>> >
>>> > This vote runs for a week and closes on 5/8/2013 at 12:00 pm. Here is
>>> my +1
>>> > for the merge.
>>> >
>>> > Regards,
>>> > Suresh
>>> >
>>>
>>
>>
>>
>> --
>> http://hortonworks.com/download/
>>
>
>
>
> --
> http://hortonworks.com/download/
>
--
http://hortonworks.com/download/
Re: [VOTE] Merge HDFS-2802 snapshot feature to trunk
Posted by Suresh Srinivas <su...@hortonworks.com>.
With 3 +1 binding votes, the vote to merge Snapshot feature from HDFS-2802
branch to trunk passes. The code will be merged soon into. The remaining
open issues related to this feature will be addressed in trunk.
Thanks to everyone who voted.
Regards,
Suresh
On Wed, May 8, 2013 at 1:12 PM, Suresh Srinivas <su...@hortonworks.com>wrote:
> Aaron, I have created HDFS-4808 to discuss the API issues you brought up.
> I agree that this needs to be resolved before merging the code to branch-2.
> Thanks for voting.
>
>
> On Tue, May 7, 2013 at 5:29 PM, Aaron T. Myers <at...@cloudera.com> wrote:
>
>> I'm +1 as well on merging this branch to trunk. I've thoroughly
>> reviewed/discussed the design and have personally reviewed much of the
>> code
>> as well. Good stuff.
>>
>> I would have preferred to wrap up the discussion regarding which API
>> should
>> include the snapshot create/delete operations before the merge, but I
>> don't
>> think that should hold up the merge to trunk. No reason we can't discuss
>> it
>> afterward, though I'd really like to come to a conclusion about that
>> before
>> we merge it to branch-2. Suresh, I believe the last action on that thread
>> was you asking Nicholas to move the discussion to a JIRA, but I don't
>> think
>> that JIRA ever got filed. Mind doing that?
>>
>>
>> --
>> Aaron T. Myers
>> Software Engineer, Cloudera
>>
>>
>> On Wed, May 1, 2013 at 11:54 AM, Suresh Srinivas <suresh@hortonworks.com
>> >wrote:
>>
>> > This is a follow up to my earlier heads up about merging Snapshot
>> feature
>> > to trunk - http://markmail.org/message/ixkyku2cebkewnzy. I am happy to
>> > announce that we have completed the development of the feature. It is
>> ready
>> > to be merged into trunk.
>> >
>> > Development of snapshot feature is tracked in the jira
>> > https://issues.apache.org/jira/browse/HDFS-2802. This is an important
>> > feature for HDFS. Please see a brief presentation that describes the
>> > feature at a highlevel from the Snapshot discussion meetup we had a
>> while
>> > back -
>> > https://issues.apache.org/jira/secure/attachment/12552861/Snapshots.pdf
>> .
>> >
>> >
>> > Details of development and testing:
>> > Development has been done in a separate branch -
>> > https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2802. The
>> > updated design is posted at -
>> >
>> >
>> https://issues.apache.org/jira/secure/attachment/12581376/Snapshots20120429.pdf
>> > .
>> > The feature development has involved close to 120 subtasks and close to
>> 25K
>> > lines of code.
>> >
>> > A lot of unit tests have been added as a part of the feature. We also
>> have
>> > been testing this in a cluster of 5 nodes with a long running test that
>> > mimics a real cluster usage with emphasis on use cases related to
>> snapshot.
>> > We are also testing non-snapshot code path running tests on a separate
>> > cluster without configuring snapshot. Please see the test plan
>> >
>> >
>> https://issues.apache.org/jira/secure/attachment/12575442/snapshot-testplan.pdf
>> > for
>> > the details. Once the feature is merged into trunk, we will continue to
>> > test and fix any bugs that may be found on the trunk.
>> >
>> > This feature is a result of many people's collaboration. The bulk of the
>> > code and design work was done by Nicholas Sze, Jing Zhao, Hari Mankude,
>> > Brandon Li, Aprit Agarwal, Sanjay Radia and me. Thanks to Ramya Sunil
>> for
>> > doing feature testing and finding many bugs. Aaron Myers, Konstantin
>> > Shvachko, Allen Wittenauer, Chris Nauroth, Todd Lipcon, Michael Stack,
>> Eli
>> > Collins, Lars Hofhansl
>> > and many others contributed to the feature definition, design and code
>> > reviews on the jiras and during meetup.
>> >
>> > This vote runs for a week and closes on 5/8/2013 at 12:00 pm. Here is
>> my +1
>> > for the merge.
>> >
>> > Regards,
>> > Suresh
>> >
>>
>
>
>
> --
> http://hortonworks.com/download/
>
--
http://hortonworks.com/download/
Re: [VOTE] Merge HDFS-2802 snapshot feature to trunk
Posted by Suresh Srinivas <su...@hortonworks.com>.
Aaron, I have created HDFS-4808 to discuss the API issues you brought up. I
agree that this needs to be resolved before merging the code to branch-2.
Thanks for voting.
On Tue, May 7, 2013 at 5:29 PM, Aaron T. Myers <at...@cloudera.com> wrote:
> I'm +1 as well on merging this branch to trunk. I've thoroughly
> reviewed/discussed the design and have personally reviewed much of the code
> as well. Good stuff.
>
> I would have preferred to wrap up the discussion regarding which API should
> include the snapshot create/delete operations before the merge, but I don't
> think that should hold up the merge to trunk. No reason we can't discuss it
> afterward, though I'd really like to come to a conclusion about that before
> we merge it to branch-2. Suresh, I believe the last action on that thread
> was you asking Nicholas to move the discussion to a JIRA, but I don't think
> that JIRA ever got filed. Mind doing that?
>
>
> --
> Aaron T. Myers
> Software Engineer, Cloudera
>
>
> On Wed, May 1, 2013 at 11:54 AM, Suresh Srinivas <suresh@hortonworks.com
> >wrote:
>
> > This is a follow up to my earlier heads up about merging Snapshot feature
> > to trunk - http://markmail.org/message/ixkyku2cebkewnzy. I am happy to
> > announce that we have completed the development of the feature. It is
> ready
> > to be merged into trunk.
> >
> > Development of snapshot feature is tracked in the jira
> > https://issues.apache.org/jira/browse/HDFS-2802. This is an important
> > feature for HDFS. Please see a brief presentation that describes the
> > feature at a highlevel from the Snapshot discussion meetup we had a while
> > back -
> > https://issues.apache.org/jira/secure/attachment/12552861/Snapshots.pdf.
> >
> >
> > Details of development and testing:
> > Development has been done in a separate branch -
> > https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2802. The
> > updated design is posted at -
> >
> >
> https://issues.apache.org/jira/secure/attachment/12581376/Snapshots20120429.pdf
> > .
> > The feature development has involved close to 120 subtasks and close to
> 25K
> > lines of code.
> >
> > A lot of unit tests have been added as a part of the feature. We also
> have
> > been testing this in a cluster of 5 nodes with a long running test that
> > mimics a real cluster usage with emphasis on use cases related to
> snapshot.
> > We are also testing non-snapshot code path running tests on a separate
> > cluster without configuring snapshot. Please see the test plan
> >
> >
> https://issues.apache.org/jira/secure/attachment/12575442/snapshot-testplan.pdf
> > for
> > the details. Once the feature is merged into trunk, we will continue to
> > test and fix any bugs that may be found on the trunk.
> >
> > This feature is a result of many people's collaboration. The bulk of the
> > code and design work was done by Nicholas Sze, Jing Zhao, Hari Mankude,
> > Brandon Li, Aprit Agarwal, Sanjay Radia and me. Thanks to Ramya Sunil
> for
> > doing feature testing and finding many bugs. Aaron Myers, Konstantin
> > Shvachko, Allen Wittenauer, Chris Nauroth, Todd Lipcon, Michael Stack,
> Eli
> > Collins, Lars Hofhansl
> > and many others contributed to the feature definition, design and code
> > reviews on the jiras and during meetup.
> >
> > This vote runs for a week and closes on 5/8/2013 at 12:00 pm. Here is my
> +1
> > for the merge.
> >
> > Regards,
> > Suresh
> >
>
--
http://hortonworks.com/download/
Re: [VOTE] Merge HDFS-2802 snapshot feature to trunk
Posted by "Aaron T. Myers" <at...@cloudera.com>.
I'm +1 as well on merging this branch to trunk. I've thoroughly
reviewed/discussed the design and have personally reviewed much of the code
as well. Good stuff.
I would have preferred to wrap up the discussion regarding which API should
include the snapshot create/delete operations before the merge, but I don't
think that should hold up the merge to trunk. No reason we can't discuss it
afterward, though I'd really like to come to a conclusion about that before
we merge it to branch-2. Suresh, I believe the last action on that thread
was you asking Nicholas to move the discussion to a JIRA, but I don't think
that JIRA ever got filed. Mind doing that?
--
Aaron T. Myers
Software Engineer, Cloudera
On Wed, May 1, 2013 at 11:54 AM, Suresh Srinivas <su...@hortonworks.com>wrote:
> This is a follow up to my earlier heads up about merging Snapshot feature
> to trunk - http://markmail.org/message/ixkyku2cebkewnzy. I am happy to
> announce that we have completed the development of the feature. It is ready
> to be merged into trunk.
>
> Development of snapshot feature is tracked in the jira
> https://issues.apache.org/jira/browse/HDFS-2802. This is an important
> feature for HDFS. Please see a brief presentation that describes the
> feature at a highlevel from the Snapshot discussion meetup we had a while
> back -
> https://issues.apache.org/jira/secure/attachment/12552861/Snapshots.pdf.
>
>
> Details of development and testing:
> Development has been done in a separate branch -
> https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2802. The
> updated design is posted at -
>
> https://issues.apache.org/jira/secure/attachment/12581376/Snapshots20120429.pdf
> .
> The feature development has involved close to 120 subtasks and close to 25K
> lines of code.
>
> A lot of unit tests have been added as a part of the feature. We also have
> been testing this in a cluster of 5 nodes with a long running test that
> mimics a real cluster usage with emphasis on use cases related to snapshot.
> We are also testing non-snapshot code path running tests on a separate
> cluster without configuring snapshot. Please see the test plan
>
> https://issues.apache.org/jira/secure/attachment/12575442/snapshot-testplan.pdf
> for
> the details. Once the feature is merged into trunk, we will continue to
> test and fix any bugs that may be found on the trunk.
>
> This feature is a result of many people's collaboration. The bulk of the
> code and design work was done by Nicholas Sze, Jing Zhao, Hari Mankude,
> Brandon Li, Aprit Agarwal, Sanjay Radia and me. Thanks to Ramya Sunil for
> doing feature testing and finding many bugs. Aaron Myers, Konstantin
> Shvachko, Allen Wittenauer, Chris Nauroth, Todd Lipcon, Michael Stack, Eli
> Collins, Lars Hofhansl
> and many others contributed to the feature definition, design and code
> reviews on the jiras and during meetup.
>
> This vote runs for a week and closes on 5/8/2013 at 12:00 pm. Here is my +1
> for the merge.
>
> Regards,
> Suresh
>
Re: [VOTE] Merge HDFS-2802 snapshot feature to trunk
Posted by Tsz Wo Sze <sz...@yahoo.com>.
+1
Tsz-Wo
________________________________
From: Suresh Srinivas <su...@hortonworks.com>
To: "hdfs-dev@hadoop.apache.org" <hd...@hadoop.apache.org>
Sent: Wednesday, May 1, 2013 11:54 AM
Subject: [VOTE] Merge HDFS-2802 snapshot feature to trunk
This is a follow up to my earlier heads up about merging Snapshot feature
to trunk - http://markmail.org/message/ixkyku2cebkewnzy. I am happy to
announce that we have completed the development of the feature. It is ready
to be merged into trunk.
Development of snapshot feature is tracked in the jira
https://issues.apache.org/jira/browse/HDFS-2802. This is an important
feature for HDFS. Please see a brief presentation that describes the
feature at a highlevel from the Snapshot discussion meetup we had a while
back -
https://issues.apache.org/jira/secure/attachment/12552861/Snapshots.pdf.
Details of development and testing:
Development has been done in a separate branch -
https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2802. The
updated design is posted at -
https://issues.apache.org/jira/secure/attachment/12581376/Snapshots20120429.pdf.
The feature development has involved close to 120 subtasks and close to 25K
lines of code.
A lot of unit tests have been added as a part of the feature. We also have
been testing this in a cluster of 5 nodes with a long running test that
mimics a real cluster usage with emphasis on use cases related to snapshot.
We are also testing non-snapshot code path running tests on a separate
cluster without configuring snapshot. Please see the test plan
https://issues.apache.org/jira/secure/attachment/12575442/snapshot-testplan.pdf
for
the details. Once the feature is merged into trunk, we will continue to
test and fix any bugs that may be found on the trunk.
This feature is a result of many people's collaboration. The bulk of the
code and design work was done by Nicholas Sze, Jing Zhao, Hari Mankude,
Brandon Li, Aprit Agarwal, Sanjay Radia and me. Thanks to Ramya Sunil for
doing feature testing and finding many bugs. Aaron Myers, Konstantin
Shvachko, Allen Wittenauer, Chris Nauroth, Todd Lipcon, Michael Stack, Eli
Collins, Lars Hofhansl
and many others contributed to the feature definition, design and code
reviews on the jiras and during meetup.
This vote runs for a week and closes on 5/8/2013 at 12:00 pm. Here is my +1
for the merge.
Regards,
Suresh