[VOTE] Merge HDFS-4949 to trunk

[Andrew Wang <an...@cloudera.com>]

Hello all,

I'd like to call a vote to merge the HDFS-4949 branch (in-memory caching)
to trunk. Colin McCabe and I have been hard at work the last 3.5 months
implementing this feature, and feel that it's reached a level of stability
and utility where it's ready for broader testing and integration.

I'd also like to thank Chris Nauroth at Hortonworks for code reviews and
bug fixes, and everyone who's reviewed the HDFS-4949 design doc and left
comments.

Obviously, I am +1 for the merge. The vote will run the standard 7 days,
closing on October 24 at 11:59PM.

Thanks,
Andrew

Re: [VOTE] Merge HDFS-4949 to trunk

[Colin McCabe <cm...@alumni.cmu.edu>]

On Thu, Oct 24, 2013 at 1:45 PM, Chris Nauroth <cn...@hortonworks.com> wrote:
> Hi Andrew,
>
> I've come to the conclusion that I'm very confused about merge votes.  :-)
>  It's not just about HDFS-4949.  I'm confused about all merge votes.
>  Rather than muddy the waters here, I've started a separate discussion on
> common-dev.
>
> I do agree with the general plan outlined here, and I will comment directly
> on the HDFS-4949 jira with a binding +1 when I see that we've completed
> that plan.

Thanks, Chris.  Andrew posted a merge patch to HDFS-4949.

We're happy that this code is getting closer to getting into trunk,
since it will make it easier to integrate with the other features in
trunk (like HDFS-2832).  There are still some follow-up tasks, but we
feel that it's easier to do those in trunk.

I'm going to update the design doc in just a moment so be sure to
check it out.  Are there any other things we should do today prior to
merging?

Colin


>
> Chris Nauroth
> Hortonworks
> http://hortonworks.com/
>
>
>
> On Wed, Oct 23, 2013 at 2:18 PM, Andrew Wang <an...@cloudera.com>wrote:
>
>> Hey Chris,
>>
>> Right now we're on track to have all of those things done by tomorrow.
>> Since the remaining issues are either not technical or do not involve major
>> changes, I was hoping we could +1 this merge vote in the spirit of "+1
>> pending jenkins". We've gotten clean unit test runs on upstream Jenkins as
>> well, so the only fixups we should need for test-patch.sh are findbugs and
>> javac (which are normally pretty trivial to clean up). Of course, all of
>> your listed prereqs and test-patch would be taken care of before actually
>> merging to trunk.
>>
>> So, we can reset the vote if you feel strongly about this, but it seems
>> like the only real result will be delaying the merge by a week.
>>
>> Thanks,
>> Andrew
>>
>>
>> On Wed, Oct 23, 2013 at 1:03 PM, Chris Nauroth <cnauroth@hortonworks.com
>> >wrote:
>>
>> > I've received some feedback that we haven't handled this merge vote the
>> > same as other comparable merge votes, and that the vote should be reset
>> > because of this.
>> >
>> > The recent custom is that we only call for the merge vote after all
>> > pre-requisites have been satisfied.  This would include committing to the
>> > feature branch all patches that the devs deem necessary before the code
>> > lands in trunk, posting a test plan, posting an updated design doc in
>> case
>> > implementation choices diverged from the original design doc, and
>> getting a
>> > good test-patch run from Jenkins on the merge patch.  This was the
>> process
>> > followed for other recent major features like HDFS-2802 (snapshots),
>> > HDFS-347 (short-circuit reads via sharing file descriptors), and
>> > HADOOP-8562 (Windows compatibility).  In this thread, we've diverged from
>> > that process by calling for a vote on a branch that hasn't yet completed
>> > the pre-requisites and stating a plan for work to be done before the
>> merge.
>> >
>> > I still support this work, but can we please restart the vote after the
>> > pre-requisites have landed in the branch?
>> >
>> > Chris Nauroth
>> > Hortonworks
>> > http://hortonworks.com/
>> >
>> >
>> >
>> > On Fri, Oct 18, 2013 at 1:37 PM, Chris Nauroth <cnauroth@hortonworks.com
>> > >wrote:
>> >
>> > > +1
>> > >
>> > > Sounds great!
>> > >
>> > > Regarding testing caching+federation, this is another thing that I had
>> > > intended to pick up as part of HDFS-5149.  I'm not sure if I can get
>> this
>> > > done in the next 7 days, so I'll keep you posted.
>> > >
>> > > Chris Nauroth
>> > > Hortonworks
>> > > http://hortonworks.com/
>> > >
>> > >
>> > >
>> > > On Fri, Oct 18, 2013 at 11:15 AM, Colin McCabe <cmccabe@alumni.cmu.edu
>> > >wrote:
>> > >
>> > >> Hi Chris,
>> > >>
>> > >> I think it's feasible to complete those tasks in the next 7 days.
>> > >> Andrew is on HDFS-5386.
>> > >>
>> > >> The test plan document is a great idea.  We'll try to get that up
>> > >> early next week.  We have a lot of unit tests now, clearly, but some
>> > >> manual testing is important too.
>> > >>
>> > >> If we discover any issues during testing, then we can push out the
>> > >> merge timeframe.  For example, one area that probably needs more
>> > >> testing is caching+federation.
>> > >>
>> > >> I would like to get HDFS-5378 and HDFS-5366 in as well.
>> > >>
>> > >> The other subtasks are "nice to have" but not really critical, and I
>> > >> think it would be just as easy to do them in trunk.  We're hoping that
>> > >> having this in trunk will make it easier for us to collaborate on
>> > >> HDFS-2832 and other ongoing work.
>> > >>
>> > >> > Also, I want to confirm that this vote only covers trunk.
>> > >> > I don't see branch-2 mentioned, so I assume that we're
>> > >> > not voting on merge to branch-2 yet.
>> > >>
>> > >> Yeah, this vote is only to merge to trunk.
>> > >>
>> > >> cheers.
>> > >> Colin
>> > >>
>> > >> On Fri, Oct 18, 2013 at 10:48 AM, Chris Nauroth
>> > >> <cn...@hortonworks.com> wrote:
>> > >> > I agree that the code has reached a stable point.  Colin and Andrew,
>> > >> thank
>> > >> > you for your contributions and collaboration.
>> > >> >
>> > >> > Throughout development, I've watched the feature grow by running
>> daily
>> > >> > builds in a pseudo-distributed deployment.  As of this week, the
>> full
>> > >> > feature set is working end-to-end.  I also think we've reached a
>> point
>> > >> of
>> > >> > API stability for clients who want to control caching
>> > programmatically.
>> > >> >
>> > >> > There are several things that I'd like to see completed before the
>> > >> merge as
>> > >> > pre-requisites:
>> > >> >
>> > >> > - HDFS-5203: Concurrent clients that add a cache directive on the
>> same
>> > >> path
>> > >> > may prematurely uncache from each other.
>> > >> > - HDFS-5385: Caching RPCs are AtMostOnce, but do not persist client
>> ID
>> > >> and
>> > >> > call ID to edit log.
>> > >> > - HDFS-5386: Add feature documentation for datanode caching.
>> > >> > - Standard clean-ups to satisfy Jenkins pre-commit on the merge
>> patch.
>> > >> >  (For example, I know we've introduced some Javadoc warnings.)
>> > >> > - Full test suite run on Windows.  (The feature is not yet
>> implemented
>> > >> on
>> > >> > Windows.  This is just intended to catch regressions.)
>> > >> > - Test plan posted to HDFS-4949, similar in scope to the snapshot
>> test
>> > >> plan
>> > >> > that was posted to HDFS-2802.  For my own part, I've run the new
>> unit
>> > >> > tests, and I've tested end-to-end in a pseudo-distributed
>> deployment.
>> > >>  It's
>> > >> > unlikely that I'll get a chance to test fully distributed before the
>> > >> vote
>> > >> > closes, so I'm curious to hear if you've done this on your side yet.
>> > >> >
>> > >> > Also, I want to confirm that this vote only covers trunk.  I don't
>> see
>> > >> > branch-2 mentioned, so I assume that we're not voting on merge to
>> > >> branch-2
>> > >> > yet.
>> > >> >
>> > >> > Before I cast my vote, can you please discuss whether or not it's
>> > >> feasible
>> > >> > to complete all of the above in the next 7 days?  For the issues
>> > >> assigned
>> > >> > to me, I do expect to complete them.
>> > >> >
>> > >> > Thanks again for all of your hard work!
>> > >> >
>> > >> > Chris Nauroth
>> > >> > Hortonworks
>> > >> > http://hortonworks.com/
>> > >> >
>> > >> >
>> > >> >
>> > >> > On Thu, Oct 17, 2013 at 3:07 PM, Colin McCabe <
>> cmccabe@alumni.cmu.edu
>> > >> >wrote:
>> > >> >
>> > >> >> +1.  Thanks, guys.
>> > >> >>
>> > >> >> best,
>> > >> >> Colin
>> > >> >>
>> > >> >> On Thu, Oct 17, 2013 at 3:01 PM, Andrew Wang <
>> > andrew.wang@cloudera.com
>> > >> >
>> > >> >> wrote:
>> > >> >> > Hello all,
>> > >> >> >
>> > >> >> > I'd like to call a vote to merge the HDFS-4949 branch (in-memory
>> > >> caching)
>> > >> >> > to trunk. Colin McCabe and I have been hard at work the last 3.5
>> > >> months
>> > >> >> > implementing this feature, and feel that it's reached a level of
>> > >> >> stability
>> > >> >> > and utility where it's ready for broader testing and integration.
>> > >> >> >
>> > >> >> > I'd also like to thank Chris Nauroth at Hortonworks for code
>> > reviews
>> > >> and
>> > >> >> > bug fixes, and everyone who's reviewed the HDFS-4949 design doc
>> and
>> > >> left
>> > >> >> > comments.
>> > >> >> >
>> > >> >> > Obviously, I am +1 for the merge. The vote will run the standard
>> 7
>> > >> days,
>> > >> >> > closing on October 24 at 11:59PM.
>> > >> >> >
>> > >> >> > Thanks,
>> > >> >> > Andrew
>> > >> >>
>> > >> >
>> > >> > --
>> > >> > CONFIDENTIALITY NOTICE
>> > >> > NOTICE: This message is intended for the use of the individual or
>> > >> entity to
>> > >> > which it is addressed and may contain information that is
>> > confidential,
>> > >> > privileged and exempt from disclosure under applicable law. If the
>> > >> reader
>> > >> > of this message is not the intended recipient, you are hereby
>> notified
>> > >> that
>> > >> > any printing, copying, dissemination, distribution, disclosure or
>> > >> > forwarding of this communication is strictly prohibited. If you have
>> > >> > received this communication in error, please contact the sender
>> > >> immediately
>> > >> > and delete it from your system. Thank You.
>> > >>
>> > >
>> > >
>> >
>> > --
>> > CONFIDENTIALITY NOTICE
>> > NOTICE: This message is intended for the use of the individual or entity
>> to
>> > which it is addressed and may contain information that is confidential,
>> > privileged and exempt from disclosure under applicable law. If the reader
>> > of this message is not the intended recipient, you are hereby notified
>> that
>> > any printing, copying, dissemination, distribution, disclosure or
>> > forwarding of this communication is strictly prohibited. If you have
>> > received this communication in error, please contact the sender
>> immediately
>> > and delete it from your system. Thank You.
>> >
>>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.

Re: [VOTE] Merge HDFS-4949 to trunk

[Chris Nauroth <cn...@hortonworks.com>]

> Are there any other things we should do today prior to merging?

Can we get the user documentation done soon (HDFS-5386)?  I've given a
round of feedback.  If it helps, I can volunteer to upload a new patch that
incorporates my feedback.

Chris Nauroth
Hortonworks
http://hortonworks.com/



On Thu, Oct 24, 2013 at 11:29 PM, Aaron T. Myers <at...@apache.org> wrote:

> I don't necessarily disagree with the general questions about the
> procedural issues of merge votes. Thanks for bringing that up in the other
> thread you mentioned. To some extent it seems like much of this has been
> based on custom, and if folks feel that more precisely defining the merge
> vote process is warranted, then I think we should take that up over on that
> thread.
>
> With regard to this particular merge vote, I've spoken with Chris offline
> about his feelings on this. He said that he is not dead-set on restarting
> the vote, though he suspects that others may be. It seems to me the
> remaining unfinished asks (e.g. updating the design doc) can reasonably be
> done either after this vote but before the merge to trunk proper, or could
> even reasonably be done after merging to trunk.
>
> Given that, I'll lend my +1 to this merge. I've been reviewing the branch
> pretty consistently since work started on it, and have personally
> run/tested several builds of it along the way. I've also reviewed the
> design thoroughly. The implementation, overall design, and API seem to me
> plenty stable enough to be merged into trunk. I know that there remains a
> handful of javac warnings in the branch that aren't in trunk, but I trust
> those will be taken care of before the merge.
>
> If anyone out there does feel strongly that this merge vote should be
> restarted, then please speak up soon. Again, we can restart the vote if
> need be, but I honestly think we'll gain very little by doing so.
>
> Best,
> Aaron
>
>
> On Fri, Oct 25, 2013 at 5:45 AM, Chris Nauroth <cnauroth@hortonworks.com
> >wrote:
>
> > Hi Andrew,
> >
> > I've come to the conclusion that I'm very confused about merge votes.
>  :-)
> >  It's not just about HDFS-4949.  I'm confused about all merge votes.
> >  Rather than muddy the waters here, I've started a separate discussion on
> > common-dev.
> >
> > I do agree with the general plan outlined here, and I will comment
> directly
> > on the HDFS-4949 jira with a binding +1 when I see that we've completed
> > that plan.
> >
> > Chris Nauroth
> > Hortonworks
> > http://hortonworks.com/
> >
> >
> >
> > On Wed, Oct 23, 2013 at 2:18 PM, Andrew Wang <andrew.wang@cloudera.com
> > >wrote:
> >
> > > Hey Chris,
> > >
> > > Right now we're on track to have all of those things done by tomorrow.
> > > Since the remaining issues are either not technical or do not involve
> > major
> > > changes, I was hoping we could +1 this merge vote in the spirit of "+1
> > > pending jenkins". We've gotten clean unit test runs on upstream Jenkins
> > as
> > > well, so the only fixups we should need for test-patch.sh are findbugs
> > and
> > > javac (which are normally pretty trivial to clean up). Of course, all
> of
> > > your listed prereqs and test-patch would be taken care of before
> actually
> > > merging to trunk.
> > >
> > > So, we can reset the vote if you feel strongly about this, but it seems
> > > like the only real result will be delaying the merge by a week.
> > >
> > > Thanks,
> > > Andrew
> > >
> > >
> > > On Wed, Oct 23, 2013 at 1:03 PM, Chris Nauroth <
> cnauroth@hortonworks.com
> > > >wrote:
> > >
> > > > I've received some feedback that we haven't handled this merge vote
> the
> > > > same as other comparable merge votes, and that the vote should be
> reset
> > > > because of this.
> > > >
> > > > The recent custom is that we only call for the merge vote after all
> > > > pre-requisites have been satisfied.  This would include committing to
> > the
> > > > feature branch all patches that the devs deem necessary before the
> code
> > > > lands in trunk, posting a test plan, posting an updated design doc in
> > > case
> > > > implementation choices diverged from the original design doc, and
> > > getting a
> > > > good test-patch run from Jenkins on the merge patch.  This was the
> > > process
> > > > followed for other recent major features like HDFS-2802 (snapshots),
> > > > HDFS-347 (short-circuit reads via sharing file descriptors), and
> > > > HADOOP-8562 (Windows compatibility).  In this thread, we've diverged
> > from
> > > > that process by calling for a vote on a branch that hasn't yet
> > completed
> > > > the pre-requisites and stating a plan for work to be done before the
> > > merge.
> > > >
> > > > I still support this work, but can we please restart the vote after
> the
> > > > pre-requisites have landed in the branch?
> > > >
> > > > Chris Nauroth
> > > > Hortonworks
> > > > http://hortonworks.com/
> > > >
> > > >
> > > >
> > > > On Fri, Oct 18, 2013 at 1:37 PM, Chris Nauroth <
> > cnauroth@hortonworks.com
> > > > >wrote:
> > > >
> > > > > +1
> > > > >
> > > > > Sounds great!
> > > > >
> > > > > Regarding testing caching+federation, this is another thing that I
> > had
> > > > > intended to pick up as part of HDFS-5149.  I'm not sure if I can
> get
> > > this
> > > > > done in the next 7 days, so I'll keep you posted.
> > > > >
> > > > > Chris Nauroth
> > > > > Hortonworks
> > > > > http://hortonworks.com/
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Oct 18, 2013 at 11:15 AM, Colin McCabe <
> > cmccabe@alumni.cmu.edu
> > > > >wrote:
> > > > >
> > > > >> Hi Chris,
> > > > >>
> > > > >> I think it's feasible to complete those tasks in the next 7 days.
> > > > >> Andrew is on HDFS-5386.
> > > > >>
> > > > >> The test plan document is a great idea.  We'll try to get that up
> > > > >> early next week.  We have a lot of unit tests now, clearly, but
> some
> > > > >> manual testing is important too.
> > > > >>
> > > > >> If we discover any issues during testing, then we can push out the
> > > > >> merge timeframe.  For example, one area that probably needs more
> > > > >> testing is caching+federation.
> > > > >>
> > > > >> I would like to get HDFS-5378 and HDFS-5366 in as well.
> > > > >>
> > > > >> The other subtasks are "nice to have" but not really critical,
> and I
> > > > >> think it would be just as easy to do them in trunk.  We're hoping
> > that
> > > > >> having this in trunk will make it easier for us to collaborate on
> > > > >> HDFS-2832 and other ongoing work.
> > > > >>
> > > > >> > Also, I want to confirm that this vote only covers trunk.
> > > > >> > I don't see branch-2 mentioned, so I assume that we're
> > > > >> > not voting on merge to branch-2 yet.
> > > > >>
> > > > >> Yeah, this vote is only to merge to trunk.
> > > > >>
> > > > >> cheers.
> > > > >> Colin
> > > > >>
> > > > >> On Fri, Oct 18, 2013 at 10:48 AM, Chris Nauroth
> > > > >> <cn...@hortonworks.com> wrote:
> > > > >> > I agree that the code has reached a stable point.  Colin and
> > Andrew,
> > > > >> thank
> > > > >> > you for your contributions and collaboration.
> > > > >> >
> > > > >> > Throughout development, I've watched the feature grow by running
> > > daily
> > > > >> > builds in a pseudo-distributed deployment.  As of this week, the
> > > full
> > > > >> > feature set is working end-to-end.  I also think we've reached a
> > > point
> > > > >> of
> > > > >> > API stability for clients who want to control caching
> > > > programmatically.
> > > > >> >
> > > > >> > There are several things that I'd like to see completed before
> the
> > > > >> merge as
> > > > >> > pre-requisites:
> > > > >> >
> > > > >> > - HDFS-5203: Concurrent clients that add a cache directive on
> the
> > > same
> > > > >> path
> > > > >> > may prematurely uncache from each other.
> > > > >> > - HDFS-5385: Caching RPCs are AtMostOnce, but do not persist
> > client
> > > ID
> > > > >> and
> > > > >> > call ID to edit log.
> > > > >> > - HDFS-5386: Add feature documentation for datanode caching.
> > > > >> > - Standard clean-ups to satisfy Jenkins pre-commit on the merge
> > > patch.
> > > > >> >  (For example, I know we've introduced some Javadoc warnings.)
> > > > >> > - Full test suite run on Windows.  (The feature is not yet
> > > implemented
> > > > >> on
> > > > >> > Windows.  This is just intended to catch regressions.)
> > > > >> > - Test plan posted to HDFS-4949, similar in scope to the
> snapshot
> > > test
> > > > >> plan
> > > > >> > that was posted to HDFS-2802.  For my own part, I've run the new
> > > unit
> > > > >> > tests, and I've tested end-to-end in a pseudo-distributed
> > > deployment.
> > > > >>  It's
> > > > >> > unlikely that I'll get a chance to test fully distributed before
> > the
> > > > >> vote
> > > > >> > closes, so I'm curious to hear if you've done this on your side
> > yet.
> > > > >> >
> > > > >> > Also, I want to confirm that this vote only covers trunk.  I
> don't
> > > see
> > > > >> > branch-2 mentioned, so I assume that we're not voting on merge
> to
> > > > >> branch-2
> > > > >> > yet.
> > > > >> >
> > > > >> > Before I cast my vote, can you please discuss whether or not
> it's
> > > > >> feasible
> > > > >> > to complete all of the above in the next 7 days?  For the issues
> > > > >> assigned
> > > > >> > to me, I do expect to complete them.
> > > > >> >
> > > > >> > Thanks again for all of your hard work!
> > > > >> >
> > > > >> > Chris Nauroth
> > > > >> > Hortonworks
> > > > >> > http://hortonworks.com/
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> > On Thu, Oct 17, 2013 at 3:07 PM, Colin McCabe <
> > > cmccabe@alumni.cmu.edu
> > > > >> >wrote:
> > > > >> >
> > > > >> >> +1.  Thanks, guys.
> > > > >> >>
> > > > >> >> best,
> > > > >> >> Colin
> > > > >> >>
> > > > >> >> On Thu, Oct 17, 2013 at 3:01 PM, Andrew Wang <
> > > > andrew.wang@cloudera.com
> > > > >> >
> > > > >> >> wrote:
> > > > >> >> > Hello all,
> > > > >> >> >
> > > > >> >> > I'd like to call a vote to merge the HDFS-4949 branch
> > (in-memory
> > > > >> caching)
> > > > >> >> > to trunk. Colin McCabe and I have been hard at work the last
> > 3.5
> > > > >> months
> > > > >> >> > implementing this feature, and feel that it's reached a level
> > of
> > > > >> >> stability
> > > > >> >> > and utility where it's ready for broader testing and
> > integration.
> > > > >> >> >
> > > > >> >> > I'd also like to thank Chris Nauroth at Hortonworks for code
> > > > reviews
> > > > >> and
> > > > >> >> > bug fixes, and everyone who's reviewed the HDFS-4949 design
> doc
> > > and
> > > > >> left
> > > > >> >> > comments.
> > > > >> >> >
> > > > >> >> > Obviously, I am +1 for the merge. The vote will run the
> > standard
> > > 7
> > > > >> days,
> > > > >> >> > closing on October 24 at 11:59PM.
> > > > >> >> >
> > > > >> >> > Thanks,
> > > > >> >> > Andrew
> > > > >> >>
> > > > >> >
> > > > >> > --
> > > > >> > CONFIDENTIALITY NOTICE
> > > > >> > NOTICE: This message is intended for the use of the individual
> or
> > > > >> entity to
> > > > >> > which it is addressed and may contain information that is
> > > > confidential,
> > > > >> > privileged and exempt from disclosure under applicable law. If
> the
> > > > >> reader
> > > > >> > of this message is not the intended recipient, you are hereby
> > > notified
> > > > >> that
> > > > >> > any printing, copying, dissemination, distribution, disclosure
> or
> > > > >> > forwarding of this communication is strictly prohibited. If you
> > have
> > > > >> > received this communication in error, please contact the sender
> > > > >> immediately
> > > > >> > and delete it from your system. Thank You.
> > > > >>
> > > > >
> > > > >
> > > >
> > > > --
> > > > CONFIDENTIALITY NOTICE
> > > > NOTICE: This message is intended for the use of the individual or
> > entity
> > > to
> > > > which it is addressed and may contain information that is
> confidential,
> > > > privileged and exempt from disclosure under applicable law. If the
> > reader
> > > > of this message is not the intended recipient, you are hereby
> notified
> > > that
> > > > any printing, copying, dissemination, distribution, disclosure or
> > > > forwarding of this communication is strictly prohibited. If you have
> > > > received this communication in error, please contact the sender
> > > immediately
> > > > and delete it from your system. Thank You.
> > > >
> > >
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: [VOTE] Merge HDFS-4949 to trunk

[Colin McCabe <cm...@alumni.cmu.edu>]

With 3 +1s, the vote passes.  Thanks, all.

best,
Colin

On Fri, Oct 25, 2013 at 4:01 PM, Colin McCabe <cm...@alumni.cmu.edu> wrote:
> On Fri, Oct 25, 2013 at 10:07 AM, Suresh Srinivas
> <su...@hortonworks.com> wrote:
>> I posted a comment in the other thread about feature branch merges.
>>
>> My preference is to make sure the requirements we have for regular patches
>> be applied to feature branch patch as well (3 +1s is the only exception).
>> Also
>> adding details about what functionality is missing (I posted a comment on
>> HDFS-4949)
>> and the changes that deferred that will be done post merge to trunk would
>> be good.
>>
>> It would be better to start the merge vote  when the work is ready instead
>> of
>> trying to optimize 1 week by doing the required work for merging in
>> parallel with
>> the vote.
>
> OK.
>
>>
>> If all the requirements for merging have been met, I am +1 on the merge,
>> without
>> the need for restarting the vote.
>>
>
> I think the requirements are all in place right now.  I'll create a
> JIRA detailing the post-merge subtasks just to make it clearer what
> the plan is from here.
>
> If there are no more comments, I'll commit later tonight.
>
> I wouldn't mind waiting a week if there was a feature someone
> absolutely felt we needed pre-merge, but I also feel like it would be
> two weeks, due to Hadoop Summit next week.
>
> best,
> Colin
>
>
>>
>> On Thu, Oct 24, 2013 at 11:29 PM, Aaron T. Myers <at...@apache.org> wrote:
>>
>>> I don't necessarily disagree with the general questions about the
>>> procedural issues of merge votes. Thanks for bringing that up in the other
>>> thread you mentioned. To some extent it seems like much of this has been
>>> based on custom, and if folks feel that more precisely defining the merge
>>> vote process is warranted, then I think we should take that up over on that
>>> thread.
>>>
>>> With regard to this particular merge vote, I've spoken with Chris offline
>>> about his feelings on this. He said that he is not dead-set on restarting
>>> the vote, though he suspects that others may be. It seems to me the
>>> remaining unfinished asks (e.g. updating the design doc) can reasonably be
>>> done either after this vote but before the merge to trunk proper, or could
>>> even reasonably be done after merging to trunk.
>>>
>>> Given that, I'll lend my +1 to this merge. I've been reviewing the branch
>>> pretty consistently since work started on it, and have personally
>>> run/tested several builds of it along the way. I've also reviewed the
>>> design thoroughly. The implementation, overall design, and API seem to me
>>> plenty stable enough to be merged into trunk. I know that there remains a
>>> handful of javac warnings in the branch that aren't in trunk, but I trust
>>> those will be taken care of before the merge.
>>>
>>> If anyone out there does feel strongly that this merge vote should be
>>> restarted, then please speak up soon. Again, we can restart the vote if
>>> need be, but I honestly think we'll gain very little by doing so.
>>>
>>> Best,
>>> Aaron
>>>
>>>
>>> On Fri, Oct 25, 2013 at 5:45 AM, Chris Nauroth <cnauroth@hortonworks.com
>>> >wrote:
>>>
>>> > Hi Andrew,
>>> >
>>> > I've come to the conclusion that I'm very confused about merge votes.
>>>  :-)
>>> >  It's not just about HDFS-4949.  I'm confused about all merge votes.
>>> >  Rather than muddy the waters here, I've started a separate discussion on
>>> > common-dev.
>>> >
>>> > I do agree with the general plan outlined here, and I will comment
>>> directly
>>> > on the HDFS-4949 jira with a binding +1 when I see that we've completed
>>> > that plan.
>>> >
>>> > Chris Nauroth
>>> > Hortonworks
>>> > http://hortonworks.com/
>>> >
>>> >
>>> >
>>> > On Wed, Oct 23, 2013 at 2:18 PM, Andrew Wang <andrew.wang@cloudera.com
>>> > >wrote:
>>> >
>>> > > Hey Chris,
>>> > >
>>> > > Right now we're on track to have all of those things done by tomorrow.
>>> > > Since the remaining issues are either not technical or do not involve
>>> > major
>>> > > changes, I was hoping we could +1 this merge vote in the spirit of "+1
>>> > > pending jenkins". We've gotten clean unit test runs on upstream Jenkins
>>> > as
>>> > > well, so the only fixups we should need for test-patch.sh are findbugs
>>> > and
>>> > > javac (which are normally pretty trivial to clean up). Of course, all
>>> of
>>> > > your listed prereqs and test-patch would be taken care of before
>>> actually
>>> > > merging to trunk.
>>> > >
>>> > > So, we can reset the vote if you feel strongly about this, but it seems
>>> > > like the only real result will be delaying the merge by a week.
>>> > >
>>> > > Thanks,
>>> > > Andrew
>>> > >
>>> > >
>>> > > On Wed, Oct 23, 2013 at 1:03 PM, Chris Nauroth <
>>> cnauroth@hortonworks.com
>>> > > >wrote:
>>> > >
>>> > > > I've received some feedback that we haven't handled this merge vote
>>> the
>>> > > > same as other comparable merge votes, and that the vote should be
>>> reset
>>> > > > because of this.
>>> > > >
>>> > > > The recent custom is that we only call for the merge vote after all
>>> > > > pre-requisites have been satisfied.  This would include committing to
>>> > the
>>> > > > feature branch all patches that the devs deem necessary before the
>>> code
>>> > > > lands in trunk, posting a test plan, posting an updated design doc in
>>> > > case
>>> > > > implementation choices diverged from the original design doc, and
>>> > > getting a
>>> > > > good test-patch run from Jenkins on the merge patch.  This was the
>>> > > process
>>> > > > followed for other recent major features like HDFS-2802 (snapshots),
>>> > > > HDFS-347 (short-circuit reads via sharing file descriptors), and
>>> > > > HADOOP-8562 (Windows compatibility).  In this thread, we've diverged
>>> > from
>>> > > > that process by calling for a vote on a branch that hasn't yet
>>> > completed
>>> > > > the pre-requisites and stating a plan for work to be done before the
>>> > > merge.
>>> > > >
>>> > > > I still support this work, but can we please restart the vote after
>>> the
>>> > > > pre-requisites have landed in the branch?
>>> > > >
>>> > > > Chris Nauroth
>>> > > > Hortonworks
>>> > > > http://hortonworks.com/
>>> > > >
>>> > > >
>>> > > >
>>> > > > On Fri, Oct 18, 2013 at 1:37 PM, Chris Nauroth <
>>> > cnauroth@hortonworks.com
>>> > > > >wrote:
>>> > > >
>>> > > > > +1
>>> > > > >
>>> > > > > Sounds great!
>>> > > > >
>>> > > > > Regarding testing caching+federation, this is another thing that I
>>> > had
>>> > > > > intended to pick up as part of HDFS-5149.  I'm not sure if I can
>>> get
>>> > > this
>>> > > > > done in the next 7 days, so I'll keep you posted.
>>> > > > >
>>> > > > > Chris Nauroth
>>> > > > > Hortonworks
>>> > > > > http://hortonworks.com/
>>> > > > >
>>> > > > >
>>> > > > >
>>> > > > > On Fri, Oct 18, 2013 at 11:15 AM, Colin McCabe <
>>> > cmccabe@alumni.cmu.edu
>>> > > > >wrote:
>>> > > > >
>>> > > > >> Hi Chris,
>>> > > > >>
>>> > > > >> I think it's feasible to complete those tasks in the next 7 days.
>>> > > > >> Andrew is on HDFS-5386.
>>> > > > >>
>>> > > > >> The test plan document is a great idea.  We'll try to get that up
>>> > > > >> early next week.  We have a lot of unit tests now, clearly, but
>>> some
>>> > > > >> manual testing is important too.
>>> > > > >>
>>> > > > >> If we discover any issues during testing, then we can push out the
>>> > > > >> merge timeframe.  For example, one area that probably needs more
>>> > > > >> testing is caching+federation.
>>> > > > >>
>>> > > > >> I would like to get HDFS-5378 and HDFS-5366 in as well.
>>> > > > >>
>>> > > > >> The other subtasks are "nice to have" but not really critical,
>>> and I
>>> > > > >> think it would be just as easy to do them in trunk.  We're hoping
>>> > that
>>> > > > >> having this in trunk will make it easier for us to collaborate on
>>> > > > >> HDFS-2832 and other ongoing work.
>>> > > > >>
>>> > > > >> > Also, I want to confirm that this vote only covers trunk.
>>> > > > >> > I don't see branch-2 mentioned, so I assume that we're
>>> > > > >> > not voting on merge to branch-2 yet.
>>> > > > >>
>>> > > > >> Yeah, this vote is only to merge to trunk.
>>> > > > >>
>>> > > > >> cheers.
>>> > > > >> Colin
>>> > > > >>
>>> > > > >> On Fri, Oct 18, 2013 at 10:48 AM, Chris Nauroth
>>> > > > >> <cn...@hortonworks.com> wrote:
>>> > > > >> > I agree that the code has reached a stable point.  Colin and
>>> > Andrew,
>>> > > > >> thank
>>> > > > >> > you for your contributions and collaboration.
>>> > > > >> >
>>> > > > >> > Throughout development, I've watched the feature grow by running
>>> > > daily
>>> > > > >> > builds in a pseudo-distributed deployment.  As of this week, the
>>> > > full
>>> > > > >> > feature set is working end-to-end.  I also think we've reached a
>>> > > point
>>> > > > >> of
>>> > > > >> > API stability for clients who want to control caching
>>> > > > programmatically.
>>> > > > >> >
>>> > > > >> > There are several things that I'd like to see completed before
>>> the
>>> > > > >> merge as
>>> > > > >> > pre-requisites:
>>> > > > >> >
>>> > > > >> > - HDFS-5203: Concurrent clients that add a cache directive on
>>> the
>>> > > same
>>> > > > >> path
>>> > > > >> > may prematurely uncache from each other.
>>> > > > >> > - HDFS-5385: Caching RPCs are AtMostOnce, but do not persist
>>> > client
>>> > > ID
>>> > > > >> and
>>> > > > >> > call ID to edit log.
>>> > > > >> > - HDFS-5386: Add feature documentation for datanode caching.
>>> > > > >> > - Standard clean-ups to satisfy Jenkins pre-commit on the merge
>>> > > patch.
>>> > > > >> >  (For example, I know we've introduced some Javadoc warnings.)
>>> > > > >> > - Full test suite run on Windows.  (The feature is not yet
>>> > > implemented
>>> > > > >> on
>>> > > > >> > Windows.  This is just intended to catch regressions.)
>>> > > > >> > - Test plan posted to HDFS-4949, similar in scope to the
>>> snapshot
>>> > > test
>>> > > > >> plan
>>> > > > >> > that was posted to HDFS-2802.  For my own part, I've run the new
>>> > > unit
>>> > > > >> > tests, and I've tested end-to-end in a pseudo-distributed
>>> > > deployment.
>>> > > > >>  It's
>>> > > > >> > unlikely that I'll get a chance to test fully distributed before
>>> > the
>>> > > > >> vote
>>> > > > >> > closes, so I'm curious to hear if you've done this on your side
>>> > yet.
>>> > > > >> >
>>> > > > >> > Also, I want to confirm that this vote only covers trunk.  I
>>> don't
>>> > > see
>>> > > > >> > branch-2 mentioned, so I assume that we're not voting on merge
>>> to
>>> > > > >> branch-2
>>> > > > >> > yet.
>>> > > > >> >
>>> > > > >> > Before I cast my vote, can you please discuss whether or not
>>> it's
>>> > > > >> feasible
>>> > > > >> > to complete all of the above in the next 7 days?  For the issues
>>> > > > >> assigned
>>> > > > >> > to me, I do expect to complete them.
>>> > > > >> >
>>> > > > >> > Thanks again for all of your hard work!
>>> > > > >> >
>>> > > > >> > Chris Nauroth
>>> > > > >> > Hortonworks
>>> > > > >> > http://hortonworks.com/
>>> > > > >> >
>>> > > > >> >
>>> > > > >> >
>>> > > > >> > On Thu, Oct 17, 2013 at 3:07 PM, Colin McCabe <
>>> > > cmccabe@alumni.cmu.edu
>>> > > > >> >wrote:
>>> > > > >> >
>>> > > > >> >> +1.  Thanks, guys.
>>> > > > >> >>
>>> > > > >> >> best,
>>> > > > >> >> Colin
>>> > > > >> >>
>>> > > > >> >> On Thu, Oct 17, 2013 at 3:01 PM, Andrew Wang <
>>> > > > andrew.wang@cloudera.com
>>> > > > >> >
>>> > > > >> >> wrote:
>>> > > > >> >> > Hello all,
>>> > > > >> >> >
>>> > > > >> >> > I'd like to call a vote to merge the HDFS-4949 branch
>>> > (in-memory
>>> > > > >> caching)
>>> > > > >> >> > to trunk. Colin McCabe and I have been hard at work the last
>>> > 3.5
>>> > > > >> months
>>> > > > >> >> > implementing this feature, and feel that it's reached a level
>>> > of
>>> > > > >> >> stability
>>> > > > >> >> > and utility where it's ready for broader testing and
>>> > integration.
>>> > > > >> >> >
>>> > > > >> >> > I'd also like to thank Chris Nauroth at Hortonworks for code
>>> > > > reviews
>>> > > > >> and
>>> > > > >> >> > bug fixes, and everyone who's reviewed the HDFS-4949 design
>>> doc
>>> > > and
>>> > > > >> left
>>> > > > >> >> > comments.
>>> > > > >> >> >
>>> > > > >> >> > Obviously, I am +1 for the merge. The vote will run the
>>> > standard
>>> > > 7
>>> > > > >> days,
>>> > > > >> >> > closing on October 24 at 11:59PM.
>>> > > > >> >> >
>>> > > > >> >> > Thanks,
>>> > > > >> >> > Andrew
>>> > > > >> >>
>>> > > > >> >
>>> > > > >> > --
>>> > > > >> > CONFIDENTIALITY NOTICE
>>> > > > >> > NOTICE: This message is intended for the use of the individual
>>> or
>>> > > > >> entity to
>>> > > > >> > which it is addressed and may contain information that is
>>> > > > confidential,
>>> > > > >> > privileged and exempt from disclosure under applicable law. If
>>> the
>>> > > > >> reader
>>> > > > >> > of this message is not the intended recipient, you are hereby
>>> > > notified
>>> > > > >> that
>>> > > > >> > any printing, copying, dissemination, distribution, disclosure
>>> or
>>> > > > >> > forwarding of this communication is strictly prohibited. If you
>>> > have
>>> > > > >> > received this communication in error, please contact the sender
>>> > > > >> immediately
>>> > > > >> > and delete it from your system. Thank You.
>>> > > > >>
>>> > > > >
>>> > > > >
>>> > > >
>>> > > > --
>>> > > > CONFIDENTIALITY NOTICE
>>> > > > NOTICE: This message is intended for the use of the individual or
>>> > entity
>>> > > to
>>> > > > which it is addressed and may contain information that is
>>> confidential,
>>> > > > privileged and exempt from disclosure under applicable law. If the
>>> > reader
>>> > > > of this message is not the intended recipient, you are hereby
>>> notified
>>> > > that
>>> > > > any printing, copying, dissemination, distribution, disclosure or
>>> > > > forwarding of this communication is strictly prohibited. If you have
>>> > > > received this communication in error, please contact the sender
>>> > > immediately
>>> > > > and delete it from your system. Thank You.
>>> > > >
>>> > >
>>> >
>>> > --
>>> > CONFIDENTIALITY NOTICE
>>> > NOTICE: This message is intended for the use of the individual or entity
>>> to
>>> > which it is addressed and may contain information that is confidential,
>>> > privileged and exempt from disclosure under applicable law. If the reader
>>> > of this message is not the intended recipient, you are hereby notified
>>> that
>>> > any printing, copying, dissemination, distribution, disclosure or
>>> > forwarding of this communication is strictly prohibited. If you have
>>> > received this communication in error, please contact the sender
>>> immediately
>>> > and delete it from your system. Thank You.
>>> >
>>>
>>
>>
>>
>> --
>> http://hortonworks.com/download/
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.

Re: [VOTE] Merge HDFS-4949 to trunk

[Colin McCabe <cm...@alumni.cmu.edu>]

On Fri, Oct 25, 2013 at 10:07 AM, Suresh Srinivas
<su...@hortonworks.com> wrote:
> I posted a comment in the other thread about feature branch merges.
>
> My preference is to make sure the requirements we have for regular patches
> be applied to feature branch patch as well (3 +1s is the only exception).
> Also
> adding details about what functionality is missing (I posted a comment on
> HDFS-4949)
> and the changes that deferred that will be done post merge to trunk would
> be good.
>
> It would be better to start the merge vote  when the work is ready instead
> of
> trying to optimize 1 week by doing the required work for merging in
> parallel with
> the vote.

OK.

>
> If all the requirements for merging have been met, I am +1 on the merge,
> without
> the need for restarting the vote.
>

I think the requirements are all in place right now.  I'll create a
JIRA detailing the post-merge subtasks just to make it clearer what
the plan is from here.

If there are no more comments, I'll commit later tonight.

I wouldn't mind waiting a week if there was a feature someone
absolutely felt we needed pre-merge, but I also feel like it would be
two weeks, due to Hadoop Summit next week.

best,
Colin


>
> On Thu, Oct 24, 2013 at 11:29 PM, Aaron T. Myers <at...@apache.org> wrote:
>
>> I don't necessarily disagree with the general questions about the
>> procedural issues of merge votes. Thanks for bringing that up in the other
>> thread you mentioned. To some extent it seems like much of this has been
>> based on custom, and if folks feel that more precisely defining the merge
>> vote process is warranted, then I think we should take that up over on that
>> thread.
>>
>> With regard to this particular merge vote, I've spoken with Chris offline
>> about his feelings on this. He said that he is not dead-set on restarting
>> the vote, though he suspects that others may be. It seems to me the
>> remaining unfinished asks (e.g. updating the design doc) can reasonably be
>> done either after this vote but before the merge to trunk proper, or could
>> even reasonably be done after merging to trunk.
>>
>> Given that, I'll lend my +1 to this merge. I've been reviewing the branch
>> pretty consistently since work started on it, and have personally
>> run/tested several builds of it along the way. I've also reviewed the
>> design thoroughly. The implementation, overall design, and API seem to me
>> plenty stable enough to be merged into trunk. I know that there remains a
>> handful of javac warnings in the branch that aren't in trunk, but I trust
>> those will be taken care of before the merge.
>>
>> If anyone out there does feel strongly that this merge vote should be
>> restarted, then please speak up soon. Again, we can restart the vote if
>> need be, but I honestly think we'll gain very little by doing so.
>>
>> Best,
>> Aaron
>>
>>
>> On Fri, Oct 25, 2013 at 5:45 AM, Chris Nauroth <cnauroth@hortonworks.com
>> >wrote:
>>
>> > Hi Andrew,
>> >
>> > I've come to the conclusion that I'm very confused about merge votes.
>>  :-)
>> >  It's not just about HDFS-4949.  I'm confused about all merge votes.
>> >  Rather than muddy the waters here, I've started a separate discussion on
>> > common-dev.
>> >
>> > I do agree with the general plan outlined here, and I will comment
>> directly
>> > on the HDFS-4949 jira with a binding +1 when I see that we've completed
>> > that plan.
>> >
>> > Chris Nauroth
>> > Hortonworks
>> > http://hortonworks.com/
>> >
>> >
>> >
>> > On Wed, Oct 23, 2013 at 2:18 PM, Andrew Wang <andrew.wang@cloudera.com
>> > >wrote:
>> >
>> > > Hey Chris,
>> > >
>> > > Right now we're on track to have all of those things done by tomorrow.
>> > > Since the remaining issues are either not technical or do not involve
>> > major
>> > > changes, I was hoping we could +1 this merge vote in the spirit of "+1
>> > > pending jenkins". We've gotten clean unit test runs on upstream Jenkins
>> > as
>> > > well, so the only fixups we should need for test-patch.sh are findbugs
>> > and
>> > > javac (which are normally pretty trivial to clean up). Of course, all
>> of
>> > > your listed prereqs and test-patch would be taken care of before
>> actually
>> > > merging to trunk.
>> > >
>> > > So, we can reset the vote if you feel strongly about this, but it seems
>> > > like the only real result will be delaying the merge by a week.
>> > >
>> > > Thanks,
>> > > Andrew
>> > >
>> > >
>> > > On Wed, Oct 23, 2013 at 1:03 PM, Chris Nauroth <
>> cnauroth@hortonworks.com
>> > > >wrote:
>> > >
>> > > > I've received some feedback that we haven't handled this merge vote
>> the
>> > > > same as other comparable merge votes, and that the vote should be
>> reset
>> > > > because of this.
>> > > >
>> > > > The recent custom is that we only call for the merge vote after all
>> > > > pre-requisites have been satisfied.  This would include committing to
>> > the
>> > > > feature branch all patches that the devs deem necessary before the
>> code
>> > > > lands in trunk, posting a test plan, posting an updated design doc in
>> > > case
>> > > > implementation choices diverged from the original design doc, and
>> > > getting a
>> > > > good test-patch run from Jenkins on the merge patch.  This was the
>> > > process
>> > > > followed for other recent major features like HDFS-2802 (snapshots),
>> > > > HDFS-347 (short-circuit reads via sharing file descriptors), and
>> > > > HADOOP-8562 (Windows compatibility).  In this thread, we've diverged
>> > from
>> > > > that process by calling for a vote on a branch that hasn't yet
>> > completed
>> > > > the pre-requisites and stating a plan for work to be done before the
>> > > merge.
>> > > >
>> > > > I still support this work, but can we please restart the vote after
>> the
>> > > > pre-requisites have landed in the branch?
>> > > >
>> > > > Chris Nauroth
>> > > > Hortonworks
>> > > > http://hortonworks.com/
>> > > >
>> > > >
>> > > >
>> > > > On Fri, Oct 18, 2013 at 1:37 PM, Chris Nauroth <
>> > cnauroth@hortonworks.com
>> > > > >wrote:
>> > > >
>> > > > > +1
>> > > > >
>> > > > > Sounds great!
>> > > > >
>> > > > > Regarding testing caching+federation, this is another thing that I
>> > had
>> > > > > intended to pick up as part of HDFS-5149.  I'm not sure if I can
>> get
>> > > this
>> > > > > done in the next 7 days, so I'll keep you posted.
>> > > > >
>> > > > > Chris Nauroth
>> > > > > Hortonworks
>> > > > > http://hortonworks.com/
>> > > > >
>> > > > >
>> > > > >
>> > > > > On Fri, Oct 18, 2013 at 11:15 AM, Colin McCabe <
>> > cmccabe@alumni.cmu.edu
>> > > > >wrote:
>> > > > >
>> > > > >> Hi Chris,
>> > > > >>
>> > > > >> I think it's feasible to complete those tasks in the next 7 days.
>> > > > >> Andrew is on HDFS-5386.
>> > > > >>
>> > > > >> The test plan document is a great idea.  We'll try to get that up
>> > > > >> early next week.  We have a lot of unit tests now, clearly, but
>> some
>> > > > >> manual testing is important too.
>> > > > >>
>> > > > >> If we discover any issues during testing, then we can push out the
>> > > > >> merge timeframe.  For example, one area that probably needs more
>> > > > >> testing is caching+federation.
>> > > > >>
>> > > > >> I would like to get HDFS-5378 and HDFS-5366 in as well.
>> > > > >>
>> > > > >> The other subtasks are "nice to have" but not really critical,
>> and I
>> > > > >> think it would be just as easy to do them in trunk.  We're hoping
>> > that
>> > > > >> having this in trunk will make it easier for us to collaborate on
>> > > > >> HDFS-2832 and other ongoing work.
>> > > > >>
>> > > > >> > Also, I want to confirm that this vote only covers trunk.
>> > > > >> > I don't see branch-2 mentioned, so I assume that we're
>> > > > >> > not voting on merge to branch-2 yet.
>> > > > >>
>> > > > >> Yeah, this vote is only to merge to trunk.
>> > > > >>
>> > > > >> cheers.
>> > > > >> Colin
>> > > > >>
>> > > > >> On Fri, Oct 18, 2013 at 10:48 AM, Chris Nauroth
>> > > > >> <cn...@hortonworks.com> wrote:
>> > > > >> > I agree that the code has reached a stable point.  Colin and
>> > Andrew,
>> > > > >> thank
>> > > > >> > you for your contributions and collaboration.
>> > > > >> >
>> > > > >> > Throughout development, I've watched the feature grow by running
>> > > daily
>> > > > >> > builds in a pseudo-distributed deployment.  As of this week, the
>> > > full
>> > > > >> > feature set is working end-to-end.  I also think we've reached a
>> > > point
>> > > > >> of
>> > > > >> > API stability for clients who want to control caching
>> > > > programmatically.
>> > > > >> >
>> > > > >> > There are several things that I'd like to see completed before
>> the
>> > > > >> merge as
>> > > > >> > pre-requisites:
>> > > > >> >
>> > > > >> > - HDFS-5203: Concurrent clients that add a cache directive on
>> the
>> > > same
>> > > > >> path
>> > > > >> > may prematurely uncache from each other.
>> > > > >> > - HDFS-5385: Caching RPCs are AtMostOnce, but do not persist
>> > client
>> > > ID
>> > > > >> and
>> > > > >> > call ID to edit log.
>> > > > >> > - HDFS-5386: Add feature documentation for datanode caching.
>> > > > >> > - Standard clean-ups to satisfy Jenkins pre-commit on the merge
>> > > patch.
>> > > > >> >  (For example, I know we've introduced some Javadoc warnings.)
>> > > > >> > - Full test suite run on Windows.  (The feature is not yet
>> > > implemented
>> > > > >> on
>> > > > >> > Windows.  This is just intended to catch regressions.)
>> > > > >> > - Test plan posted to HDFS-4949, similar in scope to the
>> snapshot
>> > > test
>> > > > >> plan
>> > > > >> > that was posted to HDFS-2802.  For my own part, I've run the new
>> > > unit
>> > > > >> > tests, and I've tested end-to-end in a pseudo-distributed
>> > > deployment.
>> > > > >>  It's
>> > > > >> > unlikely that I'll get a chance to test fully distributed before
>> > the
>> > > > >> vote
>> > > > >> > closes, so I'm curious to hear if you've done this on your side
>> > yet.
>> > > > >> >
>> > > > >> > Also, I want to confirm that this vote only covers trunk.  I
>> don't
>> > > see
>> > > > >> > branch-2 mentioned, so I assume that we're not voting on merge
>> to
>> > > > >> branch-2
>> > > > >> > yet.
>> > > > >> >
>> > > > >> > Before I cast my vote, can you please discuss whether or not
>> it's
>> > > > >> feasible
>> > > > >> > to complete all of the above in the next 7 days?  For the issues
>> > > > >> assigned
>> > > > >> > to me, I do expect to complete them.
>> > > > >> >
>> > > > >> > Thanks again for all of your hard work!
>> > > > >> >
>> > > > >> > Chris Nauroth
>> > > > >> > Hortonworks
>> > > > >> > http://hortonworks.com/
>> > > > >> >
>> > > > >> >
>> > > > >> >
>> > > > >> > On Thu, Oct 17, 2013 at 3:07 PM, Colin McCabe <
>> > > cmccabe@alumni.cmu.edu
>> > > > >> >wrote:
>> > > > >> >
>> > > > >> >> +1.  Thanks, guys.
>> > > > >> >>
>> > > > >> >> best,
>> > > > >> >> Colin
>> > > > >> >>
>> > > > >> >> On Thu, Oct 17, 2013 at 3:01 PM, Andrew Wang <
>> > > > andrew.wang@cloudera.com
>> > > > >> >
>> > > > >> >> wrote:
>> > > > >> >> > Hello all,
>> > > > >> >> >
>> > > > >> >> > I'd like to call a vote to merge the HDFS-4949 branch
>> > (in-memory
>> > > > >> caching)
>> > > > >> >> > to trunk. Colin McCabe and I have been hard at work the last
>> > 3.5
>> > > > >> months
>> > > > >> >> > implementing this feature, and feel that it's reached a level
>> > of
>> > > > >> >> stability
>> > > > >> >> > and utility where it's ready for broader testing and
>> > integration.
>> > > > >> >> >
>> > > > >> >> > I'd also like to thank Chris Nauroth at Hortonworks for code
>> > > > reviews
>> > > > >> and
>> > > > >> >> > bug fixes, and everyone who's reviewed the HDFS-4949 design
>> doc
>> > > and
>> > > > >> left
>> > > > >> >> > comments.
>> > > > >> >> >
>> > > > >> >> > Obviously, I am +1 for the merge. The vote will run the
>> > standard
>> > > 7
>> > > > >> days,
>> > > > >> >> > closing on October 24 at 11:59PM.
>> > > > >> >> >
>> > > > >> >> > Thanks,
>> > > > >> >> > Andrew
>> > > > >> >>
>> > > > >> >
>> > > > >> > --
>> > > > >> > CONFIDENTIALITY NOTICE
>> > > > >> > NOTICE: This message is intended for the use of the individual
>> or
>> > > > >> entity to
>> > > > >> > which it is addressed and may contain information that is
>> > > > confidential,
>> > > > >> > privileged and exempt from disclosure under applicable law. If
>> the
>> > > > >> reader
>> > > > >> > of this message is not the intended recipient, you are hereby
>> > > notified
>> > > > >> that
>> > > > >> > any printing, copying, dissemination, distribution, disclosure
>> or
>> > > > >> > forwarding of this communication is strictly prohibited. If you
>> > have
>> > > > >> > received this communication in error, please contact the sender
>> > > > >> immediately
>> > > > >> > and delete it from your system. Thank You.
>> > > > >>
>> > > > >
>> > > > >
>> > > >
>> > > > --
>> > > > CONFIDENTIALITY NOTICE
>> > > > NOTICE: This message is intended for the use of the individual or
>> > entity
>> > > to
>> > > > which it is addressed and may contain information that is
>> confidential,
>> > > > privileged and exempt from disclosure under applicable law. If the
>> > reader
>> > > > of this message is not the intended recipient, you are hereby
>> notified
>> > > that
>> > > > any printing, copying, dissemination, distribution, disclosure or
>> > > > forwarding of this communication is strictly prohibited. If you have
>> > > > received this communication in error, please contact the sender
>> > > immediately
>> > > > and delete it from your system. Thank You.
>> > > >
>> > >
>> >
>> > --
>> > CONFIDENTIALITY NOTICE
>> > NOTICE: This message is intended for the use of the individual or entity
>> to
>> > which it is addressed and may contain information that is confidential,
>> > privileged and exempt from disclosure under applicable law. If the reader
>> > of this message is not the intended recipient, you are hereby notified
>> that
>> > any printing, copying, dissemination, distribution, disclosure or
>> > forwarding of this communication is strictly prohibited. If you have
>> > received this communication in error, please contact the sender
>> immediately
>> > and delete it from your system. Thank You.
>> >
>>
>
>
>
> --
> http://hortonworks.com/download/
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.

Re: [VOTE] Merge HDFS-4949 to trunk

[Suresh Srinivas <su...@hortonworks.com>]

I posted a comment in the other thread about feature branch merges.

My preference is to make sure the requirements we have for regular patches
be applied to feature branch patch as well (3 +1s is the only exception).
Also
adding details about what functionality is missing (I posted a comment on
HDFS-4949)
and the changes that deferred that will be done post merge to trunk would
be good.

It would be better to start the merge vote  when the work is ready instead
of
trying to optimize 1 week by doing the required work for merging in
parallel with
the vote.

If all the requirements for merging have been met, I am +1 on the merge,
without
the need for restarting the vote.


On Thu, Oct 24, 2013 at 11:29 PM, Aaron T. Myers <at...@apache.org> wrote:

> I don't necessarily disagree with the general questions about the
> procedural issues of merge votes. Thanks for bringing that up in the other
> thread you mentioned. To some extent it seems like much of this has been
> based on custom, and if folks feel that more precisely defining the merge
> vote process is warranted, then I think we should take that up over on that
> thread.
>
> With regard to this particular merge vote, I've spoken with Chris offline
> about his feelings on this. He said that he is not dead-set on restarting
> the vote, though he suspects that others may be. It seems to me the
> remaining unfinished asks (e.g. updating the design doc) can reasonably be
> done either after this vote but before the merge to trunk proper, or could
> even reasonably be done after merging to trunk.
>
> Given that, I'll lend my +1 to this merge. I've been reviewing the branch
> pretty consistently since work started on it, and have personally
> run/tested several builds of it along the way. I've also reviewed the
> design thoroughly. The implementation, overall design, and API seem to me
> plenty stable enough to be merged into trunk. I know that there remains a
> handful of javac warnings in the branch that aren't in trunk, but I trust
> those will be taken care of before the merge.
>
> If anyone out there does feel strongly that this merge vote should be
> restarted, then please speak up soon. Again, we can restart the vote if
> need be, but I honestly think we'll gain very little by doing so.
>
> Best,
> Aaron
>
>
> On Fri, Oct 25, 2013 at 5:45 AM, Chris Nauroth <cnauroth@hortonworks.com
> >wrote:
>
> > Hi Andrew,
> >
> > I've come to the conclusion that I'm very confused about merge votes.
>  :-)
> >  It's not just about HDFS-4949.  I'm confused about all merge votes.
> >  Rather than muddy the waters here, I've started a separate discussion on
> > common-dev.
> >
> > I do agree with the general plan outlined here, and I will comment
> directly
> > on the HDFS-4949 jira with a binding +1 when I see that we've completed
> > that plan.
> >
> > Chris Nauroth
> > Hortonworks
> > http://hortonworks.com/
> >
> >
> >
> > On Wed, Oct 23, 2013 at 2:18 PM, Andrew Wang <andrew.wang@cloudera.com
> > >wrote:
> >
> > > Hey Chris,
> > >
> > > Right now we're on track to have all of those things done by tomorrow.
> > > Since the remaining issues are either not technical or do not involve
> > major
> > > changes, I was hoping we could +1 this merge vote in the spirit of "+1
> > > pending jenkins". We've gotten clean unit test runs on upstream Jenkins
> > as
> > > well, so the only fixups we should need for test-patch.sh are findbugs
> > and
> > > javac (which are normally pretty trivial to clean up). Of course, all
> of
> > > your listed prereqs and test-patch would be taken care of before
> actually
> > > merging to trunk.
> > >
> > > So, we can reset the vote if you feel strongly about this, but it seems
> > > like the only real result will be delaying the merge by a week.
> > >
> > > Thanks,
> > > Andrew
> > >
> > >
> > > On Wed, Oct 23, 2013 at 1:03 PM, Chris Nauroth <
> cnauroth@hortonworks.com
> > > >wrote:
> > >
> > > > I've received some feedback that we haven't handled this merge vote
> the
> > > > same as other comparable merge votes, and that the vote should be
> reset
> > > > because of this.
> > > >
> > > > The recent custom is that we only call for the merge vote after all
> > > > pre-requisites have been satisfied.  This would include committing to
> > the
> > > > feature branch all patches that the devs deem necessary before the
> code
> > > > lands in trunk, posting a test plan, posting an updated design doc in
> > > case
> > > > implementation choices diverged from the original design doc, and
> > > getting a
> > > > good test-patch run from Jenkins on the merge patch.  This was the
> > > process
> > > > followed for other recent major features like HDFS-2802 (snapshots),
> > > > HDFS-347 (short-circuit reads via sharing file descriptors), and
> > > > HADOOP-8562 (Windows compatibility).  In this thread, we've diverged
> > from
> > > > that process by calling for a vote on a branch that hasn't yet
> > completed
> > > > the pre-requisites and stating a plan for work to be done before the
> > > merge.
> > > >
> > > > I still support this work, but can we please restart the vote after
> the
> > > > pre-requisites have landed in the branch?
> > > >
> > > > Chris Nauroth
> > > > Hortonworks
> > > > http://hortonworks.com/
> > > >
> > > >
> > > >
> > > > On Fri, Oct 18, 2013 at 1:37 PM, Chris Nauroth <
> > cnauroth@hortonworks.com
> > > > >wrote:
> > > >
> > > > > +1
> > > > >
> > > > > Sounds great!
> > > > >
> > > > > Regarding testing caching+federation, this is another thing that I
> > had
> > > > > intended to pick up as part of HDFS-5149.  I'm not sure if I can
> get
> > > this
> > > > > done in the next 7 days, so I'll keep you posted.
> > > > >
> > > > > Chris Nauroth
> > > > > Hortonworks
> > > > > http://hortonworks.com/
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Oct 18, 2013 at 11:15 AM, Colin McCabe <
> > cmccabe@alumni.cmu.edu
> > > > >wrote:
> > > > >
> > > > >> Hi Chris,
> > > > >>
> > > > >> I think it's feasible to complete those tasks in the next 7 days.
> > > > >> Andrew is on HDFS-5386.
> > > > >>
> > > > >> The test plan document is a great idea.  We'll try to get that up
> > > > >> early next week.  We have a lot of unit tests now, clearly, but
> some
> > > > >> manual testing is important too.
> > > > >>
> > > > >> If we discover any issues during testing, then we can push out the
> > > > >> merge timeframe.  For example, one area that probably needs more
> > > > >> testing is caching+federation.
> > > > >>
> > > > >> I would like to get HDFS-5378 and HDFS-5366 in as well.
> > > > >>
> > > > >> The other subtasks are "nice to have" but not really critical,
> and I
> > > > >> think it would be just as easy to do them in trunk.  We're hoping
> > that
> > > > >> having this in trunk will make it easier for us to collaborate on
> > > > >> HDFS-2832 and other ongoing work.
> > > > >>
> > > > >> > Also, I want to confirm that this vote only covers trunk.
> > > > >> > I don't see branch-2 mentioned, so I assume that we're
> > > > >> > not voting on merge to branch-2 yet.
> > > > >>
> > > > >> Yeah, this vote is only to merge to trunk.
> > > > >>
> > > > >> cheers.
> > > > >> Colin
> > > > >>
> > > > >> On Fri, Oct 18, 2013 at 10:48 AM, Chris Nauroth
> > > > >> <cn...@hortonworks.com> wrote:
> > > > >> > I agree that the code has reached a stable point.  Colin and
> > Andrew,
> > > > >> thank
> > > > >> > you for your contributions and collaboration.
> > > > >> >
> > > > >> > Throughout development, I've watched the feature grow by running
> > > daily
> > > > >> > builds in a pseudo-distributed deployment.  As of this week, the
> > > full
> > > > >> > feature set is working end-to-end.  I also think we've reached a
> > > point
> > > > >> of
> > > > >> > API stability for clients who want to control caching
> > > > programmatically.
> > > > >> >
> > > > >> > There are several things that I'd like to see completed before
> the
> > > > >> merge as
> > > > >> > pre-requisites:
> > > > >> >
> > > > >> > - HDFS-5203: Concurrent clients that add a cache directive on
> the
> > > same
> > > > >> path
> > > > >> > may prematurely uncache from each other.
> > > > >> > - HDFS-5385: Caching RPCs are AtMostOnce, but do not persist
> > client
> > > ID
> > > > >> and
> > > > >> > call ID to edit log.
> > > > >> > - HDFS-5386: Add feature documentation for datanode caching.
> > > > >> > - Standard clean-ups to satisfy Jenkins pre-commit on the merge
> > > patch.
> > > > >> >  (For example, I know we've introduced some Javadoc warnings.)
> > > > >> > - Full test suite run on Windows.  (The feature is not yet
> > > implemented
> > > > >> on
> > > > >> > Windows.  This is just intended to catch regressions.)
> > > > >> > - Test plan posted to HDFS-4949, similar in scope to the
> snapshot
> > > test
> > > > >> plan
> > > > >> > that was posted to HDFS-2802.  For my own part, I've run the new
> > > unit
> > > > >> > tests, and I've tested end-to-end in a pseudo-distributed
> > > deployment.
> > > > >>  It's
> > > > >> > unlikely that I'll get a chance to test fully distributed before
> > the
> > > > >> vote
> > > > >> > closes, so I'm curious to hear if you've done this on your side
> > yet.
> > > > >> >
> > > > >> > Also, I want to confirm that this vote only covers trunk.  I
> don't
> > > see
> > > > >> > branch-2 mentioned, so I assume that we're not voting on merge
> to
> > > > >> branch-2
> > > > >> > yet.
> > > > >> >
> > > > >> > Before I cast my vote, can you please discuss whether or not
> it's
> > > > >> feasible
> > > > >> > to complete all of the above in the next 7 days?  For the issues
> > > > >> assigned
> > > > >> > to me, I do expect to complete them.
> > > > >> >
> > > > >> > Thanks again for all of your hard work!
> > > > >> >
> > > > >> > Chris Nauroth
> > > > >> > Hortonworks
> > > > >> > http://hortonworks.com/
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> > On Thu, Oct 17, 2013 at 3:07 PM, Colin McCabe <
> > > cmccabe@alumni.cmu.edu
> > > > >> >wrote:
> > > > >> >
> > > > >> >> +1.  Thanks, guys.
> > > > >> >>
> > > > >> >> best,
> > > > >> >> Colin
> > > > >> >>
> > > > >> >> On Thu, Oct 17, 2013 at 3:01 PM, Andrew Wang <
> > > > andrew.wang@cloudera.com
> > > > >> >
> > > > >> >> wrote:
> > > > >> >> > Hello all,
> > > > >> >> >
> > > > >> >> > I'd like to call a vote to merge the HDFS-4949 branch
> > (in-memory
> > > > >> caching)
> > > > >> >> > to trunk. Colin McCabe and I have been hard at work the last
> > 3.5
> > > > >> months
> > > > >> >> > implementing this feature, and feel that it's reached a level
> > of
> > > > >> >> stability
> > > > >> >> > and utility where it's ready for broader testing and
> > integration.
> > > > >> >> >
> > > > >> >> > I'd also like to thank Chris Nauroth at Hortonworks for code
> > > > reviews
> > > > >> and
> > > > >> >> > bug fixes, and everyone who's reviewed the HDFS-4949 design
> doc
> > > and
> > > > >> left
> > > > >> >> > comments.
> > > > >> >> >
> > > > >> >> > Obviously, I am +1 for the merge. The vote will run the
> > standard
> > > 7
> > > > >> days,
> > > > >> >> > closing on October 24 at 11:59PM.
> > > > >> >> >
> > > > >> >> > Thanks,
> > > > >> >> > Andrew
> > > > >> >>
> > > > >> >
> > > > >> > --
> > > > >> > CONFIDENTIALITY NOTICE
> > > > >> > NOTICE: This message is intended for the use of the individual
> or
> > > > >> entity to
> > > > >> > which it is addressed and may contain information that is
> > > > confidential,
> > > > >> > privileged and exempt from disclosure under applicable law. If
> the
> > > > >> reader
> > > > >> > of this message is not the intended recipient, you are hereby
> > > notified
> > > > >> that
> > > > >> > any printing, copying, dissemination, distribution, disclosure
> or
> > > > >> > forwarding of this communication is strictly prohibited. If you
> > have
> > > > >> > received this communication in error, please contact the sender
> > > > >> immediately
> > > > >> > and delete it from your system. Thank You.
> > > > >>
> > > > >
> > > > >
> > > >
> > > > --
> > > > CONFIDENTIALITY NOTICE
> > > > NOTICE: This message is intended for the use of the individual or
> > entity
> > > to
> > > > which it is addressed and may contain information that is
> confidential,
> > > > privileged and exempt from disclosure under applicable law. If the
> > reader
> > > > of this message is not the intended recipient, you are hereby
> notified
> > > that
> > > > any printing, copying, dissemination, distribution, disclosure or
> > > > forwarding of this communication is strictly prohibited. If you have
> > > > received this communication in error, please contact the sender
> > > immediately
> > > > and delete it from your system. Thank You.
> > > >
> > >
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
>



-- 
http://hortonworks.com/download/

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: [VOTE] Merge HDFS-4949 to trunk

[Suresh Srinivas <su...@hortonworks.com>]

I posted a comment in the other thread about feature branch merges.

My preference is to make sure the requirements we have for regular patches
be applied to feature branch patch as well (3 +1s is the only exception).
Also
adding details about what functionality is missing (I posted a comment on
HDFS-4949)
and the changes that deferred that will be done post merge to trunk would
be good.

It would be better to start the merge vote  when the work is ready instead
of
trying to optimize 1 week by doing the required work for merging in
parallel with
the vote.

If all the requirements for merging have been met, I am +1 on the merge,
without
the need for restarting the vote.


On Thu, Oct 24, 2013 at 11:29 PM, Aaron T. Myers <at...@apache.org> wrote:

> I don't necessarily disagree with the general questions about the
> procedural issues of merge votes. Thanks for bringing that up in the other
> thread you mentioned. To some extent it seems like much of this has been
> based on custom, and if folks feel that more precisely defining the merge
> vote process is warranted, then I think we should take that up over on that
> thread.
>
> With regard to this particular merge vote, I've spoken with Chris offline
> about his feelings on this. He said that he is not dead-set on restarting
> the vote, though he suspects that others may be. It seems to me the
> remaining unfinished asks (e.g. updating the design doc) can reasonably be
> done either after this vote but before the merge to trunk proper, or could
> even reasonably be done after merging to trunk.
>
> Given that, I'll lend my +1 to this merge. I've been reviewing the branch
> pretty consistently since work started on it, and have personally
> run/tested several builds of it along the way. I've also reviewed the
> design thoroughly. The implementation, overall design, and API seem to me
> plenty stable enough to be merged into trunk. I know that there remains a
> handful of javac warnings in the branch that aren't in trunk, but I trust
> those will be taken care of before the merge.
>
> If anyone out there does feel strongly that this merge vote should be
> restarted, then please speak up soon. Again, we can restart the vote if
> need be, but I honestly think we'll gain very little by doing so.
>
> Best,
> Aaron
>
>
> On Fri, Oct 25, 2013 at 5:45 AM, Chris Nauroth <cnauroth@hortonworks.com
> >wrote:
>
> > Hi Andrew,
> >
> > I've come to the conclusion that I'm very confused about merge votes.
>  :-)
> >  It's not just about HDFS-4949.  I'm confused about all merge votes.
> >  Rather than muddy the waters here, I've started a separate discussion on
> > common-dev.
> >
> > I do agree with the general plan outlined here, and I will comment
> directly
> > on the HDFS-4949 jira with a binding +1 when I see that we've completed
> > that plan.
> >
> > Chris Nauroth
> > Hortonworks
> > http://hortonworks.com/
> >
> >
> >
> > On Wed, Oct 23, 2013 at 2:18 PM, Andrew Wang <andrew.wang@cloudera.com
> > >wrote:
> >
> > > Hey Chris,
> > >
> > > Right now we're on track to have all of those things done by tomorrow.
> > > Since the remaining issues are either not technical or do not involve
> > major
> > > changes, I was hoping we could +1 this merge vote in the spirit of "+1
> > > pending jenkins". We've gotten clean unit test runs on upstream Jenkins
> > as
> > > well, so the only fixups we should need for test-patch.sh are findbugs
> > and
> > > javac (which are normally pretty trivial to clean up). Of course, all
> of
> > > your listed prereqs and test-patch would be taken care of before
> actually
> > > merging to trunk.
> > >
> > > So, we can reset the vote if you feel strongly about this, but it seems
> > > like the only real result will be delaying the merge by a week.
> > >
> > > Thanks,
> > > Andrew
> > >
> > >
> > > On Wed, Oct 23, 2013 at 1:03 PM, Chris Nauroth <
> cnauroth@hortonworks.com
> > > >wrote:
> > >
> > > > I've received some feedback that we haven't handled this merge vote
> the
> > > > same as other comparable merge votes, and that the vote should be
> reset
> > > > because of this.
> > > >
> > > > The recent custom is that we only call for the merge vote after all
> > > > pre-requisites have been satisfied.  This would include committing to
> > the
> > > > feature branch all patches that the devs deem necessary before the
> code
> > > > lands in trunk, posting a test plan, posting an updated design doc in
> > > case
> > > > implementation choices diverged from the original design doc, and
> > > getting a
> > > > good test-patch run from Jenkins on the merge patch.  This was the
> > > process
> > > > followed for other recent major features like HDFS-2802 (snapshots),
> > > > HDFS-347 (short-circuit reads via sharing file descriptors), and
> > > > HADOOP-8562 (Windows compatibility).  In this thread, we've diverged
> > from
> > > > that process by calling for a vote on a branch that hasn't yet
> > completed
> > > > the pre-requisites and stating a plan for work to be done before the
> > > merge.
> > > >
> > > > I still support this work, but can we please restart the vote after
> the
> > > > pre-requisites have landed in the branch?
> > > >
> > > > Chris Nauroth
> > > > Hortonworks
> > > > http://hortonworks.com/
> > > >
> > > >
> > > >
> > > > On Fri, Oct 18, 2013 at 1:37 PM, Chris Nauroth <
> > cnauroth@hortonworks.com
> > > > >wrote:
> > > >
> > > > > +1
> > > > >
> > > > > Sounds great!
> > > > >
> > > > > Regarding testing caching+federation, this is another thing that I
> > had
> > > > > intended to pick up as part of HDFS-5149.  I'm not sure if I can
> get
> > > this
> > > > > done in the next 7 days, so I'll keep you posted.
> > > > >
> > > > > Chris Nauroth
> > > > > Hortonworks
> > > > > http://hortonworks.com/
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Oct 18, 2013 at 11:15 AM, Colin McCabe <
> > cmccabe@alumni.cmu.edu
> > > > >wrote:
> > > > >
> > > > >> Hi Chris,
> > > > >>
> > > > >> I think it's feasible to complete those tasks in the next 7 days.
> > > > >> Andrew is on HDFS-5386.
> > > > >>
> > > > >> The test plan document is a great idea.  We'll try to get that up
> > > > >> early next week.  We have a lot of unit tests now, clearly, but
> some
> > > > >> manual testing is important too.
> > > > >>
> > > > >> If we discover any issues during testing, then we can push out the
> > > > >> merge timeframe.  For example, one area that probably needs more
> > > > >> testing is caching+federation.
> > > > >>
> > > > >> I would like to get HDFS-5378 and HDFS-5366 in as well.
> > > > >>
> > > > >> The other subtasks are "nice to have" but not really critical,
> and I
> > > > >> think it would be just as easy to do them in trunk.  We're hoping
> > that
> > > > >> having this in trunk will make it easier for us to collaborate on
> > > > >> HDFS-2832 and other ongoing work.
> > > > >>
> > > > >> > Also, I want to confirm that this vote only covers trunk.
> > > > >> > I don't see branch-2 mentioned, so I assume that we're
> > > > >> > not voting on merge to branch-2 yet.
> > > > >>
> > > > >> Yeah, this vote is only to merge to trunk.
> > > > >>
> > > > >> cheers.
> > > > >> Colin
> > > > >>
> > > > >> On Fri, Oct 18, 2013 at 10:48 AM, Chris Nauroth
> > > > >> <cn...@hortonworks.com> wrote:
> > > > >> > I agree that the code has reached a stable point.  Colin and
> > Andrew,
> > > > >> thank
> > > > >> > you for your contributions and collaboration.
> > > > >> >
> > > > >> > Throughout development, I've watched the feature grow by running
> > > daily
> > > > >> > builds in a pseudo-distributed deployment.  As of this week, the
> > > full
> > > > >> > feature set is working end-to-end.  I also think we've reached a
> > > point
> > > > >> of
> > > > >> > API stability for clients who want to control caching
> > > > programmatically.
> > > > >> >
> > > > >> > There are several things that I'd like to see completed before
> the
> > > > >> merge as
> > > > >> > pre-requisites:
> > > > >> >
> > > > >> > - HDFS-5203: Concurrent clients that add a cache directive on
> the
> > > same
> > > > >> path
> > > > >> > may prematurely uncache from each other.
> > > > >> > - HDFS-5385: Caching RPCs are AtMostOnce, but do not persist
> > client
> > > ID
> > > > >> and
> > > > >> > call ID to edit log.
> > > > >> > - HDFS-5386: Add feature documentation for datanode caching.
> > > > >> > - Standard clean-ups to satisfy Jenkins pre-commit on the merge
> > > patch.
> > > > >> >  (For example, I know we've introduced some Javadoc warnings.)
> > > > >> > - Full test suite run on Windows.  (The feature is not yet
> > > implemented
> > > > >> on
> > > > >> > Windows.  This is just intended to catch regressions.)
> > > > >> > - Test plan posted to HDFS-4949, similar in scope to the
> snapshot
> > > test
> > > > >> plan
> > > > >> > that was posted to HDFS-2802.  For my own part, I've run the new
> > > unit
> > > > >> > tests, and I've tested end-to-end in a pseudo-distributed
> > > deployment.
> > > > >>  It's
> > > > >> > unlikely that I'll get a chance to test fully distributed before
> > the
> > > > >> vote
> > > > >> > closes, so I'm curious to hear if you've done this on your side
> > yet.
> > > > >> >
> > > > >> > Also, I want to confirm that this vote only covers trunk.  I
> don't
> > > see
> > > > >> > branch-2 mentioned, so I assume that we're not voting on merge
> to
> > > > >> branch-2
> > > > >> > yet.
> > > > >> >
> > > > >> > Before I cast my vote, can you please discuss whether or not
> it's
> > > > >> feasible
> > > > >> > to complete all of the above in the next 7 days?  For the issues
> > > > >> assigned
> > > > >> > to me, I do expect to complete them.
> > > > >> >
> > > > >> > Thanks again for all of your hard work!
> > > > >> >
> > > > >> > Chris Nauroth
> > > > >> > Hortonworks
> > > > >> > http://hortonworks.com/
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> > On Thu, Oct 17, 2013 at 3:07 PM, Colin McCabe <
> > > cmccabe@alumni.cmu.edu
> > > > >> >wrote:
> > > > >> >
> > > > >> >> +1.  Thanks, guys.
> > > > >> >>
> > > > >> >> best,
> > > > >> >> Colin
> > > > >> >>
> > > > >> >> On Thu, Oct 17, 2013 at 3:01 PM, Andrew Wang <
> > > > andrew.wang@cloudera.com
> > > > >> >
> > > > >> >> wrote:
> > > > >> >> > Hello all,
> > > > >> >> >
> > > > >> >> > I'd like to call a vote to merge the HDFS-4949 branch
> > (in-memory
> > > > >> caching)
> > > > >> >> > to trunk. Colin McCabe and I have been hard at work the last
> > 3.5
> > > > >> months
> > > > >> >> > implementing this feature, and feel that it's reached a level
> > of
> > > > >> >> stability
> > > > >> >> > and utility where it's ready for broader testing and
> > integration.
> > > > >> >> >
> > > > >> >> > I'd also like to thank Chris Nauroth at Hortonworks for code
> > > > reviews
> > > > >> and
> > > > >> >> > bug fixes, and everyone who's reviewed the HDFS-4949 design
> doc
> > > and
> > > > >> left
> > > > >> >> > comments.
> > > > >> >> >
> > > > >> >> > Obviously, I am +1 for the merge. The vote will run the
> > standard
> > > 7
> > > > >> days,
> > > > >> >> > closing on October 24 at 11:59PM.
> > > > >> >> >
> > > > >> >> > Thanks,
> > > > >> >> > Andrew
> > > > >> >>
> > > > >> >
> > > > >> > --
> > > > >> > CONFIDENTIALITY NOTICE
> > > > >> > NOTICE: This message is intended for the use of the individual
> or
> > > > >> entity to
> > > > >> > which it is addressed and may contain information that is
> > > > confidential,
> > > > >> > privileged and exempt from disclosure under applicable law. If
> the
> > > > >> reader
> > > > >> > of this message is not the intended recipient, you are hereby
> > > notified
> > > > >> that
> > > > >> > any printing, copying, dissemination, distribution, disclosure
> or
> > > > >> > forwarding of this communication is strictly prohibited. If you
> > have
> > > > >> > received this communication in error, please contact the sender
> > > > >> immediately
> > > > >> > and delete it from your system. Thank You.
> > > > >>
> > > > >
> > > > >
> > > >
> > > > --
> > > > CONFIDENTIALITY NOTICE
> > > > NOTICE: This message is intended for the use of the individual or
> > entity
> > > to
> > > > which it is addressed and may contain information that is
> confidential,
> > > > privileged and exempt from disclosure under applicable law. If the
> > reader
> > > > of this message is not the intended recipient, you are hereby
> notified
> > > that
> > > > any printing, copying, dissemination, distribution, disclosure or
> > > > forwarding of this communication is strictly prohibited. If you have
> > > > received this communication in error, please contact the sender
> > > immediately
> > > > and delete it from your system. Thank You.
> > > >
> > >
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
>



-- 
http://hortonworks.com/download/

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: [VOTE] Merge HDFS-4949 to trunk

[Chris Nauroth <cn...@hortonworks.com>]

> Are there any other things we should do today prior to merging?

Can we get the user documentation done soon (HDFS-5386)?  I've given a
round of feedback.  If it helps, I can volunteer to upload a new patch that
incorporates my feedback.

Chris Nauroth
Hortonworks
http://hortonworks.com/



On Thu, Oct 24, 2013 at 11:29 PM, Aaron T. Myers <at...@apache.org> wrote:

> I don't necessarily disagree with the general questions about the
> procedural issues of merge votes. Thanks for bringing that up in the other
> thread you mentioned. To some extent it seems like much of this has been
> based on custom, and if folks feel that more precisely defining the merge
> vote process is warranted, then I think we should take that up over on that
> thread.
>
> With regard to this particular merge vote, I've spoken with Chris offline
> about his feelings on this. He said that he is not dead-set on restarting
> the vote, though he suspects that others may be. It seems to me the
> remaining unfinished asks (e.g. updating the design doc) can reasonably be
> done either after this vote but before the merge to trunk proper, or could
> even reasonably be done after merging to trunk.
>
> Given that, I'll lend my +1 to this merge. I've been reviewing the branch
> pretty consistently since work started on it, and have personally
> run/tested several builds of it along the way. I've also reviewed the
> design thoroughly. The implementation, overall design, and API seem to me
> plenty stable enough to be merged into trunk. I know that there remains a
> handful of javac warnings in the branch that aren't in trunk, but I trust
> those will be taken care of before the merge.
>
> If anyone out there does feel strongly that this merge vote should be
> restarted, then please speak up soon. Again, we can restart the vote if
> need be, but I honestly think we'll gain very little by doing so.
>
> Best,
> Aaron
>
>
> On Fri, Oct 25, 2013 at 5:45 AM, Chris Nauroth <cnauroth@hortonworks.com
> >wrote:
>
> > Hi Andrew,
> >
> > I've come to the conclusion that I'm very confused about merge votes.
>  :-)
> >  It's not just about HDFS-4949.  I'm confused about all merge votes.
> >  Rather than muddy the waters here, I've started a separate discussion on
> > common-dev.
> >
> > I do agree with the general plan outlined here, and I will comment
> directly
> > on the HDFS-4949 jira with a binding +1 when I see that we've completed
> > that plan.
> >
> > Chris Nauroth
> > Hortonworks
> > http://hortonworks.com/
> >
> >
> >
> > On Wed, Oct 23, 2013 at 2:18 PM, Andrew Wang <andrew.wang@cloudera.com
> > >wrote:
> >
> > > Hey Chris,
> > >
> > > Right now we're on track to have all of those things done by tomorrow.
> > > Since the remaining issues are either not technical or do not involve
> > major
> > > changes, I was hoping we could +1 this merge vote in the spirit of "+1
> > > pending jenkins". We've gotten clean unit test runs on upstream Jenkins
> > as
> > > well, so the only fixups we should need for test-patch.sh are findbugs
> > and
> > > javac (which are normally pretty trivial to clean up). Of course, all
> of
> > > your listed prereqs and test-patch would be taken care of before
> actually
> > > merging to trunk.
> > >
> > > So, we can reset the vote if you feel strongly about this, but it seems
> > > like the only real result will be delaying the merge by a week.
> > >
> > > Thanks,
> > > Andrew
> > >
> > >
> > > On Wed, Oct 23, 2013 at 1:03 PM, Chris Nauroth <
> cnauroth@hortonworks.com
> > > >wrote:
> > >
> > > > I've received some feedback that we haven't handled this merge vote
> the
> > > > same as other comparable merge votes, and that the vote should be
> reset
> > > > because of this.
> > > >
> > > > The recent custom is that we only call for the merge vote after all
> > > > pre-requisites have been satisfied.  This would include committing to
> > the
> > > > feature branch all patches that the devs deem necessary before the
> code
> > > > lands in trunk, posting a test plan, posting an updated design doc in
> > > case
> > > > implementation choices diverged from the original design doc, and
> > > getting a
> > > > good test-patch run from Jenkins on the merge patch.  This was the
> > > process
> > > > followed for other recent major features like HDFS-2802 (snapshots),
> > > > HDFS-347 (short-circuit reads via sharing file descriptors), and
> > > > HADOOP-8562 (Windows compatibility).  In this thread, we've diverged
> > from
> > > > that process by calling for a vote on a branch that hasn't yet
> > completed
> > > > the pre-requisites and stating a plan for work to be done before the
> > > merge.
> > > >
> > > > I still support this work, but can we please restart the vote after
> the
> > > > pre-requisites have landed in the branch?
> > > >
> > > > Chris Nauroth
> > > > Hortonworks
> > > > http://hortonworks.com/
> > > >
> > > >
> > > >
> > > > On Fri, Oct 18, 2013 at 1:37 PM, Chris Nauroth <
> > cnauroth@hortonworks.com
> > > > >wrote:
> > > >
> > > > > +1
> > > > >
> > > > > Sounds great!
> > > > >
> > > > > Regarding testing caching+federation, this is another thing that I
> > had
> > > > > intended to pick up as part of HDFS-5149.  I'm not sure if I can
> get
> > > this
> > > > > done in the next 7 days, so I'll keep you posted.
> > > > >
> > > > > Chris Nauroth
> > > > > Hortonworks
> > > > > http://hortonworks.com/
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Oct 18, 2013 at 11:15 AM, Colin McCabe <
> > cmccabe@alumni.cmu.edu
> > > > >wrote:
> > > > >
> > > > >> Hi Chris,
> > > > >>
> > > > >> I think it's feasible to complete those tasks in the next 7 days.
> > > > >> Andrew is on HDFS-5386.
> > > > >>
> > > > >> The test plan document is a great idea.  We'll try to get that up
> > > > >> early next week.  We have a lot of unit tests now, clearly, but
> some
> > > > >> manual testing is important too.
> > > > >>
> > > > >> If we discover any issues during testing, then we can push out the
> > > > >> merge timeframe.  For example, one area that probably needs more
> > > > >> testing is caching+federation.
> > > > >>
> > > > >> I would like to get HDFS-5378 and HDFS-5366 in as well.
> > > > >>
> > > > >> The other subtasks are "nice to have" but not really critical,
> and I
> > > > >> think it would be just as easy to do them in trunk.  We're hoping
> > that
> > > > >> having this in trunk will make it easier for us to collaborate on
> > > > >> HDFS-2832 and other ongoing work.
> > > > >>
> > > > >> > Also, I want to confirm that this vote only covers trunk.
> > > > >> > I don't see branch-2 mentioned, so I assume that we're
> > > > >> > not voting on merge to branch-2 yet.
> > > > >>
> > > > >> Yeah, this vote is only to merge to trunk.
> > > > >>
> > > > >> cheers.
> > > > >> Colin
> > > > >>
> > > > >> On Fri, Oct 18, 2013 at 10:48 AM, Chris Nauroth
> > > > >> <cn...@hortonworks.com> wrote:
> > > > >> > I agree that the code has reached a stable point.  Colin and
> > Andrew,
> > > > >> thank
> > > > >> > you for your contributions and collaboration.
> > > > >> >
> > > > >> > Throughout development, I've watched the feature grow by running
> > > daily
> > > > >> > builds in a pseudo-distributed deployment.  As of this week, the
> > > full
> > > > >> > feature set is working end-to-end.  I also think we've reached a
> > > point
> > > > >> of
> > > > >> > API stability for clients who want to control caching
> > > > programmatically.
> > > > >> >
> > > > >> > There are several things that I'd like to see completed before
> the
> > > > >> merge as
> > > > >> > pre-requisites:
> > > > >> >
> > > > >> > - HDFS-5203: Concurrent clients that add a cache directive on
> the
> > > same
> > > > >> path
> > > > >> > may prematurely uncache from each other.
> > > > >> > - HDFS-5385: Caching RPCs are AtMostOnce, but do not persist
> > client
> > > ID
> > > > >> and
> > > > >> > call ID to edit log.
> > > > >> > - HDFS-5386: Add feature documentation for datanode caching.
> > > > >> > - Standard clean-ups to satisfy Jenkins pre-commit on the merge
> > > patch.
> > > > >> >  (For example, I know we've introduced some Javadoc warnings.)
> > > > >> > - Full test suite run on Windows.  (The feature is not yet
> > > implemented
> > > > >> on
> > > > >> > Windows.  This is just intended to catch regressions.)
> > > > >> > - Test plan posted to HDFS-4949, similar in scope to the
> snapshot
> > > test
> > > > >> plan
> > > > >> > that was posted to HDFS-2802.  For my own part, I've run the new
> > > unit
> > > > >> > tests, and I've tested end-to-end in a pseudo-distributed
> > > deployment.
> > > > >>  It's
> > > > >> > unlikely that I'll get a chance to test fully distributed before
> > the
> > > > >> vote
> > > > >> > closes, so I'm curious to hear if you've done this on your side
> > yet.
> > > > >> >
> > > > >> > Also, I want to confirm that this vote only covers trunk.  I
> don't
> > > see
> > > > >> > branch-2 mentioned, so I assume that we're not voting on merge
> to
> > > > >> branch-2
> > > > >> > yet.
> > > > >> >
> > > > >> > Before I cast my vote, can you please discuss whether or not
> it's
> > > > >> feasible
> > > > >> > to complete all of the above in the next 7 days?  For the issues
> > > > >> assigned
> > > > >> > to me, I do expect to complete them.
> > > > >> >
> > > > >> > Thanks again for all of your hard work!
> > > > >> >
> > > > >> > Chris Nauroth
> > > > >> > Hortonworks
> > > > >> > http://hortonworks.com/
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> > On Thu, Oct 17, 2013 at 3:07 PM, Colin McCabe <
> > > cmccabe@alumni.cmu.edu
> > > > >> >wrote:
> > > > >> >
> > > > >> >> +1.  Thanks, guys.
> > > > >> >>
> > > > >> >> best,
> > > > >> >> Colin
> > > > >> >>
> > > > >> >> On Thu, Oct 17, 2013 at 3:01 PM, Andrew Wang <
> > > > andrew.wang@cloudera.com
> > > > >> >
> > > > >> >> wrote:
> > > > >> >> > Hello all,
> > > > >> >> >
> > > > >> >> > I'd like to call a vote to merge the HDFS-4949 branch
> > (in-memory
> > > > >> caching)
> > > > >> >> > to trunk. Colin McCabe and I have been hard at work the last
> > 3.5
> > > > >> months
> > > > >> >> > implementing this feature, and feel that it's reached a level
> > of
> > > > >> >> stability
> > > > >> >> > and utility where it's ready for broader testing and
> > integration.
> > > > >> >> >
> > > > >> >> > I'd also like to thank Chris Nauroth at Hortonworks for code
> > > > reviews
> > > > >> and
> > > > >> >> > bug fixes, and everyone who's reviewed the HDFS-4949 design
> doc
> > > and
> > > > >> left
> > > > >> >> > comments.
> > > > >> >> >
> > > > >> >> > Obviously, I am +1 for the merge. The vote will run the
> > standard
> > > 7
> > > > >> days,
> > > > >> >> > closing on October 24 at 11:59PM.
> > > > >> >> >
> > > > >> >> > Thanks,
> > > > >> >> > Andrew
> > > > >> >>
> > > > >> >
> > > > >> > --
> > > > >> > CONFIDENTIALITY NOTICE
> > > > >> > NOTICE: This message is intended for the use of the individual
> or
> > > > >> entity to
> > > > >> > which it is addressed and may contain information that is
> > > > confidential,
> > > > >> > privileged and exempt from disclosure under applicable law. If
> the
> > > > >> reader
> > > > >> > of this message is not the intended recipient, you are hereby
> > > notified
> > > > >> that
> > > > >> > any printing, copying, dissemination, distribution, disclosure
> or
> > > > >> > forwarding of this communication is strictly prohibited. If you
> > have
> > > > >> > received this communication in error, please contact the sender
> > > > >> immediately
> > > > >> > and delete it from your system. Thank You.
> > > > >>
> > > > >
> > > > >
> > > >
> > > > --
> > > > CONFIDENTIALITY NOTICE
> > > > NOTICE: This message is intended for the use of the individual or
> > entity
> > > to
> > > > which it is addressed and may contain information that is
> confidential,
> > > > privileged and exempt from disclosure under applicable law. If the
> > reader
> > > > of this message is not the intended recipient, you are hereby
> notified
> > > that
> > > > any printing, copying, dissemination, distribution, disclosure or
> > > > forwarding of this communication is strictly prohibited. If you have
> > > > received this communication in error, please contact the sender
> > > immediately
> > > > and delete it from your system. Thank You.
> > > >
> > >
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: [VOTE] Merge HDFS-4949 to trunk

["Aaron T. Myers" <at...@apache.org>]

I don't necessarily disagree with the general questions about the
procedural issues of merge votes. Thanks for bringing that up in the other
thread you mentioned. To some extent it seems like much of this has been
based on custom, and if folks feel that more precisely defining the merge
vote process is warranted, then I think we should take that up over on that
thread.

With regard to this particular merge vote, I've spoken with Chris offline
about his feelings on this. He said that he is not dead-set on restarting
the vote, though he suspects that others may be. It seems to me the
remaining unfinished asks (e.g. updating the design doc) can reasonably be
done either after this vote but before the merge to trunk proper, or could
even reasonably be done after merging to trunk.

Given that, I'll lend my +1 to this merge. I've been reviewing the branch
pretty consistently since work started on it, and have personally
run/tested several builds of it along the way. I've also reviewed the
design thoroughly. The implementation, overall design, and API seem to me
plenty stable enough to be merged into trunk. I know that there remains a
handful of javac warnings in the branch that aren't in trunk, but I trust
those will be taken care of before the merge.

If anyone out there does feel strongly that this merge vote should be
restarted, then please speak up soon. Again, we can restart the vote if
need be, but I honestly think we'll gain very little by doing so.

Best,
Aaron


On Fri, Oct 25, 2013 at 5:45 AM, Chris Nauroth <cn...@hortonworks.com>wrote:

> Hi Andrew,
>
> I've come to the conclusion that I'm very confused about merge votes.  :-)
>  It's not just about HDFS-4949.  I'm confused about all merge votes.
>  Rather than muddy the waters here, I've started a separate discussion on
> common-dev.
>
> I do agree with the general plan outlined here, and I will comment directly
> on the HDFS-4949 jira with a binding +1 when I see that we've completed
> that plan.
>
> Chris Nauroth
> Hortonworks
> http://hortonworks.com/
>
>
>
> On Wed, Oct 23, 2013 at 2:18 PM, Andrew Wang <andrew.wang@cloudera.com
> >wrote:
>
> > Hey Chris,
> >
> > Right now we're on track to have all of those things done by tomorrow.
> > Since the remaining issues are either not technical or do not involve
> major
> > changes, I was hoping we could +1 this merge vote in the spirit of "+1
> > pending jenkins". We've gotten clean unit test runs on upstream Jenkins
> as
> > well, so the only fixups we should need for test-patch.sh are findbugs
> and
> > javac (which are normally pretty trivial to clean up). Of course, all of
> > your listed prereqs and test-patch would be taken care of before actually
> > merging to trunk.
> >
> > So, we can reset the vote if you feel strongly about this, but it seems
> > like the only real result will be delaying the merge by a week.
> >
> > Thanks,
> > Andrew
> >
> >
> > On Wed, Oct 23, 2013 at 1:03 PM, Chris Nauroth <cnauroth@hortonworks.com
> > >wrote:
> >
> > > I've received some feedback that we haven't handled this merge vote the
> > > same as other comparable merge votes, and that the vote should be reset
> > > because of this.
> > >
> > > The recent custom is that we only call for the merge vote after all
> > > pre-requisites have been satisfied.  This would include committing to
> the
> > > feature branch all patches that the devs deem necessary before the code
> > > lands in trunk, posting a test plan, posting an updated design doc in
> > case
> > > implementation choices diverged from the original design doc, and
> > getting a
> > > good test-patch run from Jenkins on the merge patch.  This was the
> > process
> > > followed for other recent major features like HDFS-2802 (snapshots),
> > > HDFS-347 (short-circuit reads via sharing file descriptors), and
> > > HADOOP-8562 (Windows compatibility).  In this thread, we've diverged
> from
> > > that process by calling for a vote on a branch that hasn't yet
> completed
> > > the pre-requisites and stating a plan for work to be done before the
> > merge.
> > >
> > > I still support this work, but can we please restart the vote after the
> > > pre-requisites have landed in the branch?
> > >
> > > Chris Nauroth
> > > Hortonworks
> > > http://hortonworks.com/
> > >
> > >
> > >
> > > On Fri, Oct 18, 2013 at 1:37 PM, Chris Nauroth <
> cnauroth@hortonworks.com
> > > >wrote:
> > >
> > > > +1
> > > >
> > > > Sounds great!
> > > >
> > > > Regarding testing caching+federation, this is another thing that I
> had
> > > > intended to pick up as part of HDFS-5149.  I'm not sure if I can get
> > this
> > > > done in the next 7 days, so I'll keep you posted.
> > > >
> > > > Chris Nauroth
> > > > Hortonworks
> > > > http://hortonworks.com/
> > > >
> > > >
> > > >
> > > > On Fri, Oct 18, 2013 at 11:15 AM, Colin McCabe <
> cmccabe@alumni.cmu.edu
> > > >wrote:
> > > >
> > > >> Hi Chris,
> > > >>
> > > >> I think it's feasible to complete those tasks in the next 7 days.
> > > >> Andrew is on HDFS-5386.
> > > >>
> > > >> The test plan document is a great idea.  We'll try to get that up
> > > >> early next week.  We have a lot of unit tests now, clearly, but some
> > > >> manual testing is important too.
> > > >>
> > > >> If we discover any issues during testing, then we can push out the
> > > >> merge timeframe.  For example, one area that probably needs more
> > > >> testing is caching+federation.
> > > >>
> > > >> I would like to get HDFS-5378 and HDFS-5366 in as well.
> > > >>
> > > >> The other subtasks are "nice to have" but not really critical, and I
> > > >> think it would be just as easy to do them in trunk.  We're hoping
> that
> > > >> having this in trunk will make it easier for us to collaborate on
> > > >> HDFS-2832 and other ongoing work.
> > > >>
> > > >> > Also, I want to confirm that this vote only covers trunk.
> > > >> > I don't see branch-2 mentioned, so I assume that we're
> > > >> > not voting on merge to branch-2 yet.
> > > >>
> > > >> Yeah, this vote is only to merge to trunk.
> > > >>
> > > >> cheers.
> > > >> Colin
> > > >>
> > > >> On Fri, Oct 18, 2013 at 10:48 AM, Chris Nauroth
> > > >> <cn...@hortonworks.com> wrote:
> > > >> > I agree that the code has reached a stable point.  Colin and
> Andrew,
> > > >> thank
> > > >> > you for your contributions and collaboration.
> > > >> >
> > > >> > Throughout development, I've watched the feature grow by running
> > daily
> > > >> > builds in a pseudo-distributed deployment.  As of this week, the
> > full
> > > >> > feature set is working end-to-end.  I also think we've reached a
> > point
> > > >> of
> > > >> > API stability for clients who want to control caching
> > > programmatically.
> > > >> >
> > > >> > There are several things that I'd like to see completed before the
> > > >> merge as
> > > >> > pre-requisites:
> > > >> >
> > > >> > - HDFS-5203: Concurrent clients that add a cache directive on the
> > same
> > > >> path
> > > >> > may prematurely uncache from each other.
> > > >> > - HDFS-5385: Caching RPCs are AtMostOnce, but do not persist
> client
> > ID
> > > >> and
> > > >> > call ID to edit log.
> > > >> > - HDFS-5386: Add feature documentation for datanode caching.
> > > >> > - Standard clean-ups to satisfy Jenkins pre-commit on the merge
> > patch.
> > > >> >  (For example, I know we've introduced some Javadoc warnings.)
> > > >> > - Full test suite run on Windows.  (The feature is not yet
> > implemented
> > > >> on
> > > >> > Windows.  This is just intended to catch regressions.)
> > > >> > - Test plan posted to HDFS-4949, similar in scope to the snapshot
> > test
> > > >> plan
> > > >> > that was posted to HDFS-2802.  For my own part, I've run the new
> > unit
> > > >> > tests, and I've tested end-to-end in a pseudo-distributed
> > deployment.
> > > >>  It's
> > > >> > unlikely that I'll get a chance to test fully distributed before
> the
> > > >> vote
> > > >> > closes, so I'm curious to hear if you've done this on your side
> yet.
> > > >> >
> > > >> > Also, I want to confirm that this vote only covers trunk.  I don't
> > see
> > > >> > branch-2 mentioned, so I assume that we're not voting on merge to
> > > >> branch-2
> > > >> > yet.
> > > >> >
> > > >> > Before I cast my vote, can you please discuss whether or not it's
> > > >> feasible
> > > >> > to complete all of the above in the next 7 days?  For the issues
> > > >> assigned
> > > >> > to me, I do expect to complete them.
> > > >> >
> > > >> > Thanks again for all of your hard work!
> > > >> >
> > > >> > Chris Nauroth
> > > >> > Hortonworks
> > > >> > http://hortonworks.com/
> > > >> >
> > > >> >
> > > >> >
> > > >> > On Thu, Oct 17, 2013 at 3:07 PM, Colin McCabe <
> > cmccabe@alumni.cmu.edu
> > > >> >wrote:
> > > >> >
> > > >> >> +1.  Thanks, guys.
> > > >> >>
> > > >> >> best,
> > > >> >> Colin
> > > >> >>
> > > >> >> On Thu, Oct 17, 2013 at 3:01 PM, Andrew Wang <
> > > andrew.wang@cloudera.com
> > > >> >
> > > >> >> wrote:
> > > >> >> > Hello all,
> > > >> >> >
> > > >> >> > I'd like to call a vote to merge the HDFS-4949 branch
> (in-memory
> > > >> caching)
> > > >> >> > to trunk. Colin McCabe and I have been hard at work the last
> 3.5
> > > >> months
> > > >> >> > implementing this feature, and feel that it's reached a level
> of
> > > >> >> stability
> > > >> >> > and utility where it's ready for broader testing and
> integration.
> > > >> >> >
> > > >> >> > I'd also like to thank Chris Nauroth at Hortonworks for code
> > > reviews
> > > >> and
> > > >> >> > bug fixes, and everyone who's reviewed the HDFS-4949 design doc
> > and
> > > >> left
> > > >> >> > comments.
> > > >> >> >
> > > >> >> > Obviously, I am +1 for the merge. The vote will run the
> standard
> > 7
> > > >> days,
> > > >> >> > closing on October 24 at 11:59PM.
> > > >> >> >
> > > >> >> > Thanks,
> > > >> >> > Andrew
> > > >> >>
> > > >> >
> > > >> > --
> > > >> > CONFIDENTIALITY NOTICE
> > > >> > NOTICE: This message is intended for the use of the individual or
> > > >> entity to
> > > >> > which it is addressed and may contain information that is
> > > confidential,
> > > >> > privileged and exempt from disclosure under applicable law. If the
> > > >> reader
> > > >> > of this message is not the intended recipient, you are hereby
> > notified
> > > >> that
> > > >> > any printing, copying, dissemination, distribution, disclosure or
> > > >> > forwarding of this communication is strictly prohibited. If you
> have
> > > >> > received this communication in error, please contact the sender
> > > >> immediately
> > > >> > and delete it from your system. Thank You.
> > > >>
> > > >
> > > >
> > >
> > > --
> > > CONFIDENTIALITY NOTICE
> > > NOTICE: This message is intended for the use of the individual or
> entity
> > to
> > > which it is addressed and may contain information that is confidential,
> > > privileged and exempt from disclosure under applicable law. If the
> reader
> > > of this message is not the intended recipient, you are hereby notified
> > that
> > > any printing, copying, dissemination, distribution, disclosure or
> > > forwarding of this communication is strictly prohibited. If you have
> > > received this communication in error, please contact the sender
> > immediately
> > > and delete it from your system. Thank You.
> > >
> >
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Re: [VOTE] Merge HDFS-4949 to trunk

["Aaron T. Myers" <at...@apache.org>]

I don't necessarily disagree with the general questions about the
procedural issues of merge votes. Thanks for bringing that up in the other
thread you mentioned. To some extent it seems like much of this has been
based on custom, and if folks feel that more precisely defining the merge
vote process is warranted, then I think we should take that up over on that
thread.

With regard to this particular merge vote, I've spoken with Chris offline
about his feelings on this. He said that he is not dead-set on restarting
the vote, though he suspects that others may be. It seems to me the
remaining unfinished asks (e.g. updating the design doc) can reasonably be
done either after this vote but before the merge to trunk proper, or could
even reasonably be done after merging to trunk.

Given that, I'll lend my +1 to this merge. I've been reviewing the branch
pretty consistently since work started on it, and have personally
run/tested several builds of it along the way. I've also reviewed the
design thoroughly. The implementation, overall design, and API seem to me
plenty stable enough to be merged into trunk. I know that there remains a
handful of javac warnings in the branch that aren't in trunk, but I trust
those will be taken care of before the merge.

If anyone out there does feel strongly that this merge vote should be
restarted, then please speak up soon. Again, we can restart the vote if
need be, but I honestly think we'll gain very little by doing so.

Best,
Aaron


On Fri, Oct 25, 2013 at 5:45 AM, Chris Nauroth <cn...@hortonworks.com>wrote:

> Hi Andrew,
>
> I've come to the conclusion that I'm very confused about merge votes.  :-)
>  It's not just about HDFS-4949.  I'm confused about all merge votes.
>  Rather than muddy the waters here, I've started a separate discussion on
> common-dev.
>
> I do agree with the general plan outlined here, and I will comment directly
> on the HDFS-4949 jira with a binding +1 when I see that we've completed
> that plan.
>
> Chris Nauroth
> Hortonworks
> http://hortonworks.com/
>
>
>
> On Wed, Oct 23, 2013 at 2:18 PM, Andrew Wang <andrew.wang@cloudera.com
> >wrote:
>
> > Hey Chris,
> >
> > Right now we're on track to have all of those things done by tomorrow.
> > Since the remaining issues are either not technical or do not involve
> major
> > changes, I was hoping we could +1 this merge vote in the spirit of "+1
> > pending jenkins". We've gotten clean unit test runs on upstream Jenkins
> as
> > well, so the only fixups we should need for test-patch.sh are findbugs
> and
> > javac (which are normally pretty trivial to clean up). Of course, all of
> > your listed prereqs and test-patch would be taken care of before actually
> > merging to trunk.
> >
> > So, we can reset the vote if you feel strongly about this, but it seems
> > like the only real result will be delaying the merge by a week.
> >
> > Thanks,
> > Andrew
> >
> >
> > On Wed, Oct 23, 2013 at 1:03 PM, Chris Nauroth <cnauroth@hortonworks.com
> > >wrote:
> >
> > > I've received some feedback that we haven't handled this merge vote the
> > > same as other comparable merge votes, and that the vote should be reset
> > > because of this.
> > >
> > > The recent custom is that we only call for the merge vote after all
> > > pre-requisites have been satisfied.  This would include committing to
> the
> > > feature branch all patches that the devs deem necessary before the code
> > > lands in trunk, posting a test plan, posting an updated design doc in
> > case
> > > implementation choices diverged from the original design doc, and
> > getting a
> > > good test-patch run from Jenkins on the merge patch.  This was the
> > process
> > > followed for other recent major features like HDFS-2802 (snapshots),
> > > HDFS-347 (short-circuit reads via sharing file descriptors), and
> > > HADOOP-8562 (Windows compatibility).  In this thread, we've diverged
> from
> > > that process by calling for a vote on a branch that hasn't yet
> completed
> > > the pre-requisites and stating a plan for work to be done before the
> > merge.
> > >
> > > I still support this work, but can we please restart the vote after the
> > > pre-requisites have landed in the branch?
> > >
> > > Chris Nauroth
> > > Hortonworks
> > > http://hortonworks.com/
> > >
> > >
> > >
> > > On Fri, Oct 18, 2013 at 1:37 PM, Chris Nauroth <
> cnauroth@hortonworks.com
> > > >wrote:
> > >
> > > > +1
> > > >
> > > > Sounds great!
> > > >
> > > > Regarding testing caching+federation, this is another thing that I
> had
> > > > intended to pick up as part of HDFS-5149.  I'm not sure if I can get
> > this
> > > > done in the next 7 days, so I'll keep you posted.
> > > >
> > > > Chris Nauroth
> > > > Hortonworks
> > > > http://hortonworks.com/
> > > >
> > > >
> > > >
> > > > On Fri, Oct 18, 2013 at 11:15 AM, Colin McCabe <
> cmccabe@alumni.cmu.edu
> > > >wrote:
> > > >
> > > >> Hi Chris,
> > > >>
> > > >> I think it's feasible to complete those tasks in the next 7 days.
> > > >> Andrew is on HDFS-5386.
> > > >>
> > > >> The test plan document is a great idea.  We'll try to get that up
> > > >> early next week.  We have a lot of unit tests now, clearly, but some
> > > >> manual testing is important too.
> > > >>
> > > >> If we discover any issues during testing, then we can push out the
> > > >> merge timeframe.  For example, one area that probably needs more
> > > >> testing is caching+federation.
> > > >>
> > > >> I would like to get HDFS-5378 and HDFS-5366 in as well.
> > > >>
> > > >> The other subtasks are "nice to have" but not really critical, and I
> > > >> think it would be just as easy to do them in trunk.  We're hoping
> that
> > > >> having this in trunk will make it easier for us to collaborate on
> > > >> HDFS-2832 and other ongoing work.
> > > >>
> > > >> > Also, I want to confirm that this vote only covers trunk.
> > > >> > I don't see branch-2 mentioned, so I assume that we're
> > > >> > not voting on merge to branch-2 yet.
> > > >>
> > > >> Yeah, this vote is only to merge to trunk.
> > > >>
> > > >> cheers.
> > > >> Colin
> > > >>
> > > >> On Fri, Oct 18, 2013 at 10:48 AM, Chris Nauroth
> > > >> <cn...@hortonworks.com> wrote:
> > > >> > I agree that the code has reached a stable point.  Colin and
> Andrew,
> > > >> thank
> > > >> > you for your contributions and collaboration.
> > > >> >
> > > >> > Throughout development, I've watched the feature grow by running
> > daily
> > > >> > builds in a pseudo-distributed deployment.  As of this week, the
> > full
> > > >> > feature set is working end-to-end.  I also think we've reached a
> > point
> > > >> of
> > > >> > API stability for clients who want to control caching
> > > programmatically.
> > > >> >
> > > >> > There are several things that I'd like to see completed before the
> > > >> merge as
> > > >> > pre-requisites:
> > > >> >
> > > >> > - HDFS-5203: Concurrent clients that add a cache directive on the
> > same
> > > >> path
> > > >> > may prematurely uncache from each other.
> > > >> > - HDFS-5385: Caching RPCs are AtMostOnce, but do not persist
> client
> > ID
> > > >> and
> > > >> > call ID to edit log.
> > > >> > - HDFS-5386: Add feature documentation for datanode caching.
> > > >> > - Standard clean-ups to satisfy Jenkins pre-commit on the merge
> > patch.
> > > >> >  (For example, I know we've introduced some Javadoc warnings.)
> > > >> > - Full test suite run on Windows.  (The feature is not yet
> > implemented
> > > >> on
> > > >> > Windows.  This is just intended to catch regressions.)
> > > >> > - Test plan posted to HDFS-4949, similar in scope to the snapshot
> > test
> > > >> plan
> > > >> > that was posted to HDFS-2802.  For my own part, I've run the new
> > unit
> > > >> > tests, and I've tested end-to-end in a pseudo-distributed
> > deployment.
> > > >>  It's
> > > >> > unlikely that I'll get a chance to test fully distributed before
> the
> > > >> vote
> > > >> > closes, so I'm curious to hear if you've done this on your side
> yet.
> > > >> >
> > > >> > Also, I want to confirm that this vote only covers trunk.  I don't
> > see
> > > >> > branch-2 mentioned, so I assume that we're not voting on merge to
> > > >> branch-2
> > > >> > yet.
> > > >> >
> > > >> > Before I cast my vote, can you please discuss whether or not it's
> > > >> feasible
> > > >> > to complete all of the above in the next 7 days?  For the issues
> > > >> assigned
> > > >> > to me, I do expect to complete them.
> > > >> >
> > > >> > Thanks again for all of your hard work!
> > > >> >
> > > >> > Chris Nauroth
> > > >> > Hortonworks
> > > >> > http://hortonworks.com/
> > > >> >
> > > >> >
> > > >> >
> > > >> > On Thu, Oct 17, 2013 at 3:07 PM, Colin McCabe <
> > cmccabe@alumni.cmu.edu
> > > >> >wrote:
> > > >> >
> > > >> >> +1.  Thanks, guys.
> > > >> >>
> > > >> >> best,
> > > >> >> Colin
> > > >> >>
> > > >> >> On Thu, Oct 17, 2013 at 3:01 PM, Andrew Wang <
> > > andrew.wang@cloudera.com
> > > >> >
> > > >> >> wrote:
> > > >> >> > Hello all,
> > > >> >> >
> > > >> >> > I'd like to call a vote to merge the HDFS-4949 branch
> (in-memory
> > > >> caching)
> > > >> >> > to trunk. Colin McCabe and I have been hard at work the last
> 3.5
> > > >> months
> > > >> >> > implementing this feature, and feel that it's reached a level
> of
> > > >> >> stability
> > > >> >> > and utility where it's ready for broader testing and
> integration.
> > > >> >> >
> > > >> >> > I'd also like to thank Chris Nauroth at Hortonworks for code
> > > reviews
> > > >> and
> > > >> >> > bug fixes, and everyone who's reviewed the HDFS-4949 design doc
> > and
> > > >> left
> > > >> >> > comments.
> > > >> >> >
> > > >> >> > Obviously, I am +1 for the merge. The vote will run the
> standard
> > 7
> > > >> days,
> > > >> >> > closing on October 24 at 11:59PM.
> > > >> >> >
> > > >> >> > Thanks,
> > > >> >> > Andrew
> > > >> >>
> > > >> >
> > > >> > --
> > > >> > CONFIDENTIALITY NOTICE
> > > >> > NOTICE: This message is intended for the use of the individual or
> > > >> entity to
> > > >> > which it is addressed and may contain information that is
> > > confidential,
> > > >> > privileged and exempt from disclosure under applicable law. If the
> > > >> reader
> > > >> > of this message is not the intended recipient, you are hereby
> > notified
> > > >> that
> > > >> > any printing, copying, dissemination, distribution, disclosure or
> > > >> > forwarding of this communication is strictly prohibited. If you
> have
> > > >> > received this communication in error, please contact the sender
> > > >> immediately
> > > >> > and delete it from your system. Thank You.
> > > >>
> > > >
> > > >
> > >
> > > --
> > > CONFIDENTIALITY NOTICE
> > > NOTICE: This message is intended for the use of the individual or
> entity
> > to
> > > which it is addressed and may contain information that is confidential,
> > > privileged and exempt from disclosure under applicable law. If the
> reader
> > > of this message is not the intended recipient, you are hereby notified
> > that
> > > any printing, copying, dissemination, distribution, disclosure or
> > > forwarding of this communication is strictly prohibited. If you have
> > > received this communication in error, please contact the sender
> > immediately
> > > and delete it from your system. Thank You.
> > >
> >
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Re: [VOTE] Merge HDFS-4949 to trunk

[Colin McCabe <cm...@alumni.cmu.edu>]

On Thu, Oct 24, 2013 at 1:45 PM, Chris Nauroth <cn...@hortonworks.com> wrote:
> Hi Andrew,
>
> I've come to the conclusion that I'm very confused about merge votes.  :-)
>  It's not just about HDFS-4949.  I'm confused about all merge votes.
>  Rather than muddy the waters here, I've started a separate discussion on
> common-dev.
>
> I do agree with the general plan outlined here, and I will comment directly
> on the HDFS-4949 jira with a binding +1 when I see that we've completed
> that plan.

Thanks, Chris.  Andrew posted a merge patch to HDFS-4949.

We're happy that this code is getting closer to getting into trunk,
since it will make it easier to integrate with the other features in
trunk (like HDFS-2832).  There are still some follow-up tasks, but we
feel that it's easier to do those in trunk.

I'm going to update the design doc in just a moment so be sure to
check it out.  Are there any other things we should do today prior to
merging?

Colin


>
> Chris Nauroth
> Hortonworks
> http://hortonworks.com/
>
>
>
> On Wed, Oct 23, 2013 at 2:18 PM, Andrew Wang <an...@cloudera.com>wrote:
>
>> Hey Chris,
>>
>> Right now we're on track to have all of those things done by tomorrow.
>> Since the remaining issues are either not technical or do not involve major
>> changes, I was hoping we could +1 this merge vote in the spirit of "+1
>> pending jenkins". We've gotten clean unit test runs on upstream Jenkins as
>> well, so the only fixups we should need for test-patch.sh are findbugs and
>> javac (which are normally pretty trivial to clean up). Of course, all of
>> your listed prereqs and test-patch would be taken care of before actually
>> merging to trunk.
>>
>> So, we can reset the vote if you feel strongly about this, but it seems
>> like the only real result will be delaying the merge by a week.
>>
>> Thanks,
>> Andrew
>>
>>
>> On Wed, Oct 23, 2013 at 1:03 PM, Chris Nauroth <cnauroth@hortonworks.com
>> >wrote:
>>
>> > I've received some feedback that we haven't handled this merge vote the
>> > same as other comparable merge votes, and that the vote should be reset
>> > because of this.
>> >
>> > The recent custom is that we only call for the merge vote after all
>> > pre-requisites have been satisfied.  This would include committing to the
>> > feature branch all patches that the devs deem necessary before the code
>> > lands in trunk, posting a test plan, posting an updated design doc in
>> case
>> > implementation choices diverged from the original design doc, and
>> getting a
>> > good test-patch run from Jenkins on the merge patch.  This was the
>> process
>> > followed for other recent major features like HDFS-2802 (snapshots),
>> > HDFS-347 (short-circuit reads via sharing file descriptors), and
>> > HADOOP-8562 (Windows compatibility).  In this thread, we've diverged from
>> > that process by calling for a vote on a branch that hasn't yet completed
>> > the pre-requisites and stating a plan for work to be done before the
>> merge.
>> >
>> > I still support this work, but can we please restart the vote after the
>> > pre-requisites have landed in the branch?
>> >
>> > Chris Nauroth
>> > Hortonworks
>> > http://hortonworks.com/
>> >
>> >
>> >
>> > On Fri, Oct 18, 2013 at 1:37 PM, Chris Nauroth <cnauroth@hortonworks.com
>> > >wrote:
>> >
>> > > +1
>> > >
>> > > Sounds great!
>> > >
>> > > Regarding testing caching+federation, this is another thing that I had
>> > > intended to pick up as part of HDFS-5149.  I'm not sure if I can get
>> this
>> > > done in the next 7 days, so I'll keep you posted.
>> > >
>> > > Chris Nauroth
>> > > Hortonworks
>> > > http://hortonworks.com/
>> > >
>> > >
>> > >
>> > > On Fri, Oct 18, 2013 at 11:15 AM, Colin McCabe <cmccabe@alumni.cmu.edu
>> > >wrote:
>> > >
>> > >> Hi Chris,
>> > >>
>> > >> I think it's feasible to complete those tasks in the next 7 days.
>> > >> Andrew is on HDFS-5386.
>> > >>
>> > >> The test plan document is a great idea.  We'll try to get that up
>> > >> early next week.  We have a lot of unit tests now, clearly, but some
>> > >> manual testing is important too.
>> > >>
>> > >> If we discover any issues during testing, then we can push out the
>> > >> merge timeframe.  For example, one area that probably needs more
>> > >> testing is caching+federation.
>> > >>
>> > >> I would like to get HDFS-5378 and HDFS-5366 in as well.
>> > >>
>> > >> The other subtasks are "nice to have" but not really critical, and I
>> > >> think it would be just as easy to do them in trunk.  We're hoping that
>> > >> having this in trunk will make it easier for us to collaborate on
>> > >> HDFS-2832 and other ongoing work.
>> > >>
>> > >> > Also, I want to confirm that this vote only covers trunk.
>> > >> > I don't see branch-2 mentioned, so I assume that we're
>> > >> > not voting on merge to branch-2 yet.
>> > >>
>> > >> Yeah, this vote is only to merge to trunk.
>> > >>
>> > >> cheers.
>> > >> Colin
>> > >>
>> > >> On Fri, Oct 18, 2013 at 10:48 AM, Chris Nauroth
>> > >> <cn...@hortonworks.com> wrote:
>> > >> > I agree that the code has reached a stable point.  Colin and Andrew,
>> > >> thank
>> > >> > you for your contributions and collaboration.
>> > >> >
>> > >> > Throughout development, I've watched the feature grow by running
>> daily
>> > >> > builds in a pseudo-distributed deployment.  As of this week, the
>> full
>> > >> > feature set is working end-to-end.  I also think we've reached a
>> point
>> > >> of
>> > >> > API stability for clients who want to control caching
>> > programmatically.
>> > >> >
>> > >> > There are several things that I'd like to see completed before the
>> > >> merge as
>> > >> > pre-requisites:
>> > >> >
>> > >> > - HDFS-5203: Concurrent clients that add a cache directive on the
>> same
>> > >> path
>> > >> > may prematurely uncache from each other.
>> > >> > - HDFS-5385: Caching RPCs are AtMostOnce, but do not persist client
>> ID
>> > >> and
>> > >> > call ID to edit log.
>> > >> > - HDFS-5386: Add feature documentation for datanode caching.
>> > >> > - Standard clean-ups to satisfy Jenkins pre-commit on the merge
>> patch.
>> > >> >  (For example, I know we've introduced some Javadoc warnings.)
>> > >> > - Full test suite run on Windows.  (The feature is not yet
>> implemented
>> > >> on
>> > >> > Windows.  This is just intended to catch regressions.)
>> > >> > - Test plan posted to HDFS-4949, similar in scope to the snapshot
>> test
>> > >> plan
>> > >> > that was posted to HDFS-2802.  For my own part, I've run the new
>> unit
>> > >> > tests, and I've tested end-to-end in a pseudo-distributed
>> deployment.
>> > >>  It's
>> > >> > unlikely that I'll get a chance to test fully distributed before the
>> > >> vote
>> > >> > closes, so I'm curious to hear if you've done this on your side yet.
>> > >> >
>> > >> > Also, I want to confirm that this vote only covers trunk.  I don't
>> see
>> > >> > branch-2 mentioned, so I assume that we're not voting on merge to
>> > >> branch-2
>> > >> > yet.
>> > >> >
>> > >> > Before I cast my vote, can you please discuss whether or not it's
>> > >> feasible
>> > >> > to complete all of the above in the next 7 days?  For the issues
>> > >> assigned
>> > >> > to me, I do expect to complete them.
>> > >> >
>> > >> > Thanks again for all of your hard work!
>> > >> >
>> > >> > Chris Nauroth
>> > >> > Hortonworks
>> > >> > http://hortonworks.com/
>> > >> >
>> > >> >
>> > >> >
>> > >> > On Thu, Oct 17, 2013 at 3:07 PM, Colin McCabe <
>> cmccabe@alumni.cmu.edu
>> > >> >wrote:
>> > >> >
>> > >> >> +1.  Thanks, guys.
>> > >> >>
>> > >> >> best,
>> > >> >> Colin
>> > >> >>
>> > >> >> On Thu, Oct 17, 2013 at 3:01 PM, Andrew Wang <
>> > andrew.wang@cloudera.com
>> > >> >
>> > >> >> wrote:
>> > >> >> > Hello all,
>> > >> >> >
>> > >> >> > I'd like to call a vote to merge the HDFS-4949 branch (in-memory
>> > >> caching)
>> > >> >> > to trunk. Colin McCabe and I have been hard at work the last 3.5
>> > >> months
>> > >> >> > implementing this feature, and feel that it's reached a level of
>> > >> >> stability
>> > >> >> > and utility where it's ready for broader testing and integration.
>> > >> >> >
>> > >> >> > I'd also like to thank Chris Nauroth at Hortonworks for code
>> > reviews
>> > >> and
>> > >> >> > bug fixes, and everyone who's reviewed the HDFS-4949 design doc
>> and
>> > >> left
>> > >> >> > comments.
>> > >> >> >
>> > >> >> > Obviously, I am +1 for the merge. The vote will run the standard
>> 7
>> > >> days,
>> > >> >> > closing on October 24 at 11:59PM.
>> > >> >> >
>> > >> >> > Thanks,
>> > >> >> > Andrew
>> > >> >>
>> > >> >
>> > >> > --
>> > >> > CONFIDENTIALITY NOTICE
>> > >> > NOTICE: This message is intended for the use of the individual or
>> > >> entity to
>> > >> > which it is addressed and may contain information that is
>> > confidential,
>> > >> > privileged and exempt from disclosure under applicable law. If the
>> > >> reader
>> > >> > of this message is not the intended recipient, you are hereby
>> notified
>> > >> that
>> > >> > any printing, copying, dissemination, distribution, disclosure or
>> > >> > forwarding of this communication is strictly prohibited. If you have
>> > >> > received this communication in error, please contact the sender
>> > >> immediately
>> > >> > and delete it from your system. Thank You.
>> > >>
>> > >
>> > >
>> >
>> > --
>> > CONFIDENTIALITY NOTICE
>> > NOTICE: This message is intended for the use of the individual or entity
>> to
>> > which it is addressed and may contain information that is confidential,
>> > privileged and exempt from disclosure under applicable law. If the reader
>> > of this message is not the intended recipient, you are hereby notified
>> that
>> > any printing, copying, dissemination, distribution, disclosure or
>> > forwarding of this communication is strictly prohibited. If you have
>> > received this communication in error, please contact the sender
>> immediately
>> > and delete it from your system. Thank You.
>> >
>>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.

Re: [VOTE] Merge HDFS-4949 to trunk

[Chris Nauroth <cn...@hortonworks.com>]

Hi Andrew,

I've come to the conclusion that I'm very confused about merge votes.  :-)
 It's not just about HDFS-4949.  I'm confused about all merge votes.
 Rather than muddy the waters here, I've started a separate discussion on
common-dev.

I do agree with the general plan outlined here, and I will comment directly
on the HDFS-4949 jira with a binding +1 when I see that we've completed
that plan.

Chris Nauroth
Hortonworks
http://hortonworks.com/



On Wed, Oct 23, 2013 at 2:18 PM, Andrew Wang <an...@cloudera.com>wrote:

> Hey Chris,
>
> Right now we're on track to have all of those things done by tomorrow.
> Since the remaining issues are either not technical or do not involve major
> changes, I was hoping we could +1 this merge vote in the spirit of "+1
> pending jenkins". We've gotten clean unit test runs on upstream Jenkins as
> well, so the only fixups we should need for test-patch.sh are findbugs and
> javac (which are normally pretty trivial to clean up). Of course, all of
> your listed prereqs and test-patch would be taken care of before actually
> merging to trunk.
>
> So, we can reset the vote if you feel strongly about this, but it seems
> like the only real result will be delaying the merge by a week.
>
> Thanks,
> Andrew
>
>
> On Wed, Oct 23, 2013 at 1:03 PM, Chris Nauroth <cnauroth@hortonworks.com
> >wrote:
>
> > I've received some feedback that we haven't handled this merge vote the
> > same as other comparable merge votes, and that the vote should be reset
> > because of this.
> >
> > The recent custom is that we only call for the merge vote after all
> > pre-requisites have been satisfied.  This would include committing to the
> > feature branch all patches that the devs deem necessary before the code
> > lands in trunk, posting a test plan, posting an updated design doc in
> case
> > implementation choices diverged from the original design doc, and
> getting a
> > good test-patch run from Jenkins on the merge patch.  This was the
> process
> > followed for other recent major features like HDFS-2802 (snapshots),
> > HDFS-347 (short-circuit reads via sharing file descriptors), and
> > HADOOP-8562 (Windows compatibility).  In this thread, we've diverged from
> > that process by calling for a vote on a branch that hasn't yet completed
> > the pre-requisites and stating a plan for work to be done before the
> merge.
> >
> > I still support this work, but can we please restart the vote after the
> > pre-requisites have landed in the branch?
> >
> > Chris Nauroth
> > Hortonworks
> > http://hortonworks.com/
> >
> >
> >
> > On Fri, Oct 18, 2013 at 1:37 PM, Chris Nauroth <cnauroth@hortonworks.com
> > >wrote:
> >
> > > +1
> > >
> > > Sounds great!
> > >
> > > Regarding testing caching+federation, this is another thing that I had
> > > intended to pick up as part of HDFS-5149.  I'm not sure if I can get
> this
> > > done in the next 7 days, so I'll keep you posted.
> > >
> > > Chris Nauroth
> > > Hortonworks
> > > http://hortonworks.com/
> > >
> > >
> > >
> > > On Fri, Oct 18, 2013 at 11:15 AM, Colin McCabe <cmccabe@alumni.cmu.edu
> > >wrote:
> > >
> > >> Hi Chris,
> > >>
> > >> I think it's feasible to complete those tasks in the next 7 days.
> > >> Andrew is on HDFS-5386.
> > >>
> > >> The test plan document is a great idea.  We'll try to get that up
> > >> early next week.  We have a lot of unit tests now, clearly, but some
> > >> manual testing is important too.
> > >>
> > >> If we discover any issues during testing, then we can push out the
> > >> merge timeframe.  For example, one area that probably needs more
> > >> testing is caching+federation.
> > >>
> > >> I would like to get HDFS-5378 and HDFS-5366 in as well.
> > >>
> > >> The other subtasks are "nice to have" but not really critical, and I
> > >> think it would be just as easy to do them in trunk.  We're hoping that
> > >> having this in trunk will make it easier for us to collaborate on
> > >> HDFS-2832 and other ongoing work.
> > >>
> > >> > Also, I want to confirm that this vote only covers trunk.
> > >> > I don't see branch-2 mentioned, so I assume that we're
> > >> > not voting on merge to branch-2 yet.
> > >>
> > >> Yeah, this vote is only to merge to trunk.
> > >>
> > >> cheers.
> > >> Colin
> > >>
> > >> On Fri, Oct 18, 2013 at 10:48 AM, Chris Nauroth
> > >> <cn...@hortonworks.com> wrote:
> > >> > I agree that the code has reached a stable point.  Colin and Andrew,
> > >> thank
> > >> > you for your contributions and collaboration.
> > >> >
> > >> > Throughout development, I've watched the feature grow by running
> daily
> > >> > builds in a pseudo-distributed deployment.  As of this week, the
> full
> > >> > feature set is working end-to-end.  I also think we've reached a
> point
> > >> of
> > >> > API stability for clients who want to control caching
> > programmatically.
> > >> >
> > >> > There are several things that I'd like to see completed before the
> > >> merge as
> > >> > pre-requisites:
> > >> >
> > >> > - HDFS-5203: Concurrent clients that add a cache directive on the
> same
> > >> path
> > >> > may prematurely uncache from each other.
> > >> > - HDFS-5385: Caching RPCs are AtMostOnce, but do not persist client
> ID
> > >> and
> > >> > call ID to edit log.
> > >> > - HDFS-5386: Add feature documentation for datanode caching.
> > >> > - Standard clean-ups to satisfy Jenkins pre-commit on the merge
> patch.
> > >> >  (For example, I know we've introduced some Javadoc warnings.)
> > >> > - Full test suite run on Windows.  (The feature is not yet
> implemented
> > >> on
> > >> > Windows.  This is just intended to catch regressions.)
> > >> > - Test plan posted to HDFS-4949, similar in scope to the snapshot
> test
> > >> plan
> > >> > that was posted to HDFS-2802.  For my own part, I've run the new
> unit
> > >> > tests, and I've tested end-to-end in a pseudo-distributed
> deployment.
> > >>  It's
> > >> > unlikely that I'll get a chance to test fully distributed before the
> > >> vote
> > >> > closes, so I'm curious to hear if you've done this on your side yet.
> > >> >
> > >> > Also, I want to confirm that this vote only covers trunk.  I don't
> see
> > >> > branch-2 mentioned, so I assume that we're not voting on merge to
> > >> branch-2
> > >> > yet.
> > >> >
> > >> > Before I cast my vote, can you please discuss whether or not it's
> > >> feasible
> > >> > to complete all of the above in the next 7 days?  For the issues
> > >> assigned
> > >> > to me, I do expect to complete them.
> > >> >
> > >> > Thanks again for all of your hard work!
> > >> >
> > >> > Chris Nauroth
> > >> > Hortonworks
> > >> > http://hortonworks.com/
> > >> >
> > >> >
> > >> >
> > >> > On Thu, Oct 17, 2013 at 3:07 PM, Colin McCabe <
> cmccabe@alumni.cmu.edu
> > >> >wrote:
> > >> >
> > >> >> +1.  Thanks, guys.
> > >> >>
> > >> >> best,
> > >> >> Colin
> > >> >>
> > >> >> On Thu, Oct 17, 2013 at 3:01 PM, Andrew Wang <
> > andrew.wang@cloudera.com
> > >> >
> > >> >> wrote:
> > >> >> > Hello all,
> > >> >> >
> > >> >> > I'd like to call a vote to merge the HDFS-4949 branch (in-memory
> > >> caching)
> > >> >> > to trunk. Colin McCabe and I have been hard at work the last 3.5
> > >> months
> > >> >> > implementing this feature, and feel that it's reached a level of
> > >> >> stability
> > >> >> > and utility where it's ready for broader testing and integration.
> > >> >> >
> > >> >> > I'd also like to thank Chris Nauroth at Hortonworks for code
> > reviews
> > >> and
> > >> >> > bug fixes, and everyone who's reviewed the HDFS-4949 design doc
> and
> > >> left
> > >> >> > comments.
> > >> >> >
> > >> >> > Obviously, I am +1 for the merge. The vote will run the standard
> 7
> > >> days,
> > >> >> > closing on October 24 at 11:59PM.
> > >> >> >
> > >> >> > Thanks,
> > >> >> > Andrew
> > >> >>
> > >> >
> > >> > --
> > >> > CONFIDENTIALITY NOTICE
> > >> > NOTICE: This message is intended for the use of the individual or
> > >> entity to
> > >> > which it is addressed and may contain information that is
> > confidential,
> > >> > privileged and exempt from disclosure under applicable law. If the
> > >> reader
> > >> > of this message is not the intended recipient, you are hereby
> notified
> > >> that
> > >> > any printing, copying, dissemination, distribution, disclosure or
> > >> > forwarding of this communication is strictly prohibited. If you have
> > >> > received this communication in error, please contact the sender
> > >> immediately
> > >> > and delete it from your system. Thank You.
> > >>
> > >
> > >
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: [VOTE] Merge HDFS-4949 to trunk

["Aaron T. Myers" <at...@apache.org>]

On Thu, Oct 24, 2013 at 6:18 AM, Andrew Wang <an...@cloudera.com>wrote:

> Right now we're on track to have all of those things done by tomorrow.
> Since the remaining issues are either not technical or do not involve major
> changes, I was hoping we could +1 this merge vote in the spirit of "+1
> pending jenkins". We've gotten clean unit test runs on upstream Jenkins as
> well, so the only fixups we should need for test-patch.sh are findbugs and
> javac (which are normally pretty trivial to clean up). Of course, all of
> your listed prereqs and test-patch would be taken care of before actually
> merging to trunk.
>
> So, we can reset the vote if you feel strongly about this, but it seems
> like the only real result will be delaying the merge by a week.
>

I agree with this. Chris raised some concerns 6 days ago, but seems like
these have all been addressed since then. Resetting the vote would seem to
serve little purpose except to delay the merge by another week. If the
merge vote were to be restarted, I'd expect that we'd quickly see the
requisite three +1s be cast, and then we'd wait around for 7 days.

Chris, does this make sense to you? Appreciate a prompt response since I
believe this vote is supposed to close at midnight tonight.

Thanks folks.

Best,
Aaron

Re: [VOTE] Merge HDFS-4949 to trunk

["Aaron T. Myers" <at...@apache.org>]

On Thu, Oct 24, 2013 at 6:18 AM, Andrew Wang <an...@cloudera.com>wrote:

> Right now we're on track to have all of those things done by tomorrow.
> Since the remaining issues are either not technical or do not involve major
> changes, I was hoping we could +1 this merge vote in the spirit of "+1
> pending jenkins". We've gotten clean unit test runs on upstream Jenkins as
> well, so the only fixups we should need for test-patch.sh are findbugs and
> javac (which are normally pretty trivial to clean up). Of course, all of
> your listed prereqs and test-patch would be taken care of before actually
> merging to trunk.
>
> So, we can reset the vote if you feel strongly about this, but it seems
> like the only real result will be delaying the merge by a week.
>

I agree with this. Chris raised some concerns 6 days ago, but seems like
these have all been addressed since then. Resetting the vote would seem to
serve little purpose except to delay the merge by another week. If the
merge vote were to be restarted, I'd expect that we'd quickly see the
requisite three +1s be cast, and then we'd wait around for 7 days.

Chris, does this make sense to you? Appreciate a prompt response since I
believe this vote is supposed to close at midnight tonight.

Thanks folks.

Best,
Aaron

Re: [VOTE] Merge HDFS-4949 to trunk

[Chris Nauroth <cn...@hortonworks.com>]

Hi Andrew,

I've come to the conclusion that I'm very confused about merge votes.  :-)
 It's not just about HDFS-4949.  I'm confused about all merge votes.
 Rather than muddy the waters here, I've started a separate discussion on
common-dev.

I do agree with the general plan outlined here, and I will comment directly
on the HDFS-4949 jira with a binding +1 when I see that we've completed
that plan.

Chris Nauroth
Hortonworks
http://hortonworks.com/



On Wed, Oct 23, 2013 at 2:18 PM, Andrew Wang <an...@cloudera.com>wrote:

> Hey Chris,
>
> Right now we're on track to have all of those things done by tomorrow.
> Since the remaining issues are either not technical or do not involve major
> changes, I was hoping we could +1 this merge vote in the spirit of "+1
> pending jenkins". We've gotten clean unit test runs on upstream Jenkins as
> well, so the only fixups we should need for test-patch.sh are findbugs and
> javac (which are normally pretty trivial to clean up). Of course, all of
> your listed prereqs and test-patch would be taken care of before actually
> merging to trunk.
>
> So, we can reset the vote if you feel strongly about this, but it seems
> like the only real result will be delaying the merge by a week.
>
> Thanks,
> Andrew
>
>
> On Wed, Oct 23, 2013 at 1:03 PM, Chris Nauroth <cnauroth@hortonworks.com
> >wrote:
>
> > I've received some feedback that we haven't handled this merge vote the
> > same as other comparable merge votes, and that the vote should be reset
> > because of this.
> >
> > The recent custom is that we only call for the merge vote after all
> > pre-requisites have been satisfied.  This would include committing to the
> > feature branch all patches that the devs deem necessary before the code
> > lands in trunk, posting a test plan, posting an updated design doc in
> case
> > implementation choices diverged from the original design doc, and
> getting a
> > good test-patch run from Jenkins on the merge patch.  This was the
> process
> > followed for other recent major features like HDFS-2802 (snapshots),
> > HDFS-347 (short-circuit reads via sharing file descriptors), and
> > HADOOP-8562 (Windows compatibility).  In this thread, we've diverged from
> > that process by calling for a vote on a branch that hasn't yet completed
> > the pre-requisites and stating a plan for work to be done before the
> merge.
> >
> > I still support this work, but can we please restart the vote after the
> > pre-requisites have landed in the branch?
> >
> > Chris Nauroth
> > Hortonworks
> > http://hortonworks.com/
> >
> >
> >
> > On Fri, Oct 18, 2013 at 1:37 PM, Chris Nauroth <cnauroth@hortonworks.com
> > >wrote:
> >
> > > +1
> > >
> > > Sounds great!
> > >
> > > Regarding testing caching+federation, this is another thing that I had
> > > intended to pick up as part of HDFS-5149.  I'm not sure if I can get
> this
> > > done in the next 7 days, so I'll keep you posted.
> > >
> > > Chris Nauroth
> > > Hortonworks
> > > http://hortonworks.com/
> > >
> > >
> > >
> > > On Fri, Oct 18, 2013 at 11:15 AM, Colin McCabe <cmccabe@alumni.cmu.edu
> > >wrote:
> > >
> > >> Hi Chris,
> > >>
> > >> I think it's feasible to complete those tasks in the next 7 days.
> > >> Andrew is on HDFS-5386.
> > >>
> > >> The test plan document is a great idea.  We'll try to get that up
> > >> early next week.  We have a lot of unit tests now, clearly, but some
> > >> manual testing is important too.
> > >>
> > >> If we discover any issues during testing, then we can push out the
> > >> merge timeframe.  For example, one area that probably needs more
> > >> testing is caching+federation.
> > >>
> > >> I would like to get HDFS-5378 and HDFS-5366 in as well.
> > >>
> > >> The other subtasks are "nice to have" but not really critical, and I
> > >> think it would be just as easy to do them in trunk.  We're hoping that
> > >> having this in trunk will make it easier for us to collaborate on
> > >> HDFS-2832 and other ongoing work.
> > >>
> > >> > Also, I want to confirm that this vote only covers trunk.
> > >> > I don't see branch-2 mentioned, so I assume that we're
> > >> > not voting on merge to branch-2 yet.
> > >>
> > >> Yeah, this vote is only to merge to trunk.
> > >>
> > >> cheers.
> > >> Colin
> > >>
> > >> On Fri, Oct 18, 2013 at 10:48 AM, Chris Nauroth
> > >> <cn...@hortonworks.com> wrote:
> > >> > I agree that the code has reached a stable point.  Colin and Andrew,
> > >> thank
> > >> > you for your contributions and collaboration.
> > >> >
> > >> > Throughout development, I've watched the feature grow by running
> daily
> > >> > builds in a pseudo-distributed deployment.  As of this week, the
> full
> > >> > feature set is working end-to-end.  I also think we've reached a
> point
> > >> of
> > >> > API stability for clients who want to control caching
> > programmatically.
> > >> >
> > >> > There are several things that I'd like to see completed before the
> > >> merge as
> > >> > pre-requisites:
> > >> >
> > >> > - HDFS-5203: Concurrent clients that add a cache directive on the
> same
> > >> path
> > >> > may prematurely uncache from each other.
> > >> > - HDFS-5385: Caching RPCs are AtMostOnce, but do not persist client
> ID
> > >> and
> > >> > call ID to edit log.
> > >> > - HDFS-5386: Add feature documentation for datanode caching.
> > >> > - Standard clean-ups to satisfy Jenkins pre-commit on the merge
> patch.
> > >> >  (For example, I know we've introduced some Javadoc warnings.)
> > >> > - Full test suite run on Windows.  (The feature is not yet
> implemented
> > >> on
> > >> > Windows.  This is just intended to catch regressions.)
> > >> > - Test plan posted to HDFS-4949, similar in scope to the snapshot
> test
> > >> plan
> > >> > that was posted to HDFS-2802.  For my own part, I've run the new
> unit
> > >> > tests, and I've tested end-to-end in a pseudo-distributed
> deployment.
> > >>  It's
> > >> > unlikely that I'll get a chance to test fully distributed before the
> > >> vote
> > >> > closes, so I'm curious to hear if you've done this on your side yet.
> > >> >
> > >> > Also, I want to confirm that this vote only covers trunk.  I don't
> see
> > >> > branch-2 mentioned, so I assume that we're not voting on merge to
> > >> branch-2
> > >> > yet.
> > >> >
> > >> > Before I cast my vote, can you please discuss whether or not it's
> > >> feasible
> > >> > to complete all of the above in the next 7 days?  For the issues
> > >> assigned
> > >> > to me, I do expect to complete them.
> > >> >
> > >> > Thanks again for all of your hard work!
> > >> >
> > >> > Chris Nauroth
> > >> > Hortonworks
> > >> > http://hortonworks.com/
> > >> >
> > >> >
> > >> >
> > >> > On Thu, Oct 17, 2013 at 3:07 PM, Colin McCabe <
> cmccabe@alumni.cmu.edu
> > >> >wrote:
> > >> >
> > >> >> +1.  Thanks, guys.
> > >> >>
> > >> >> best,
> > >> >> Colin
> > >> >>
> > >> >> On Thu, Oct 17, 2013 at 3:01 PM, Andrew Wang <
> > andrew.wang@cloudera.com
> > >> >
> > >> >> wrote:
> > >> >> > Hello all,
> > >> >> >
> > >> >> > I'd like to call a vote to merge the HDFS-4949 branch (in-memory
> > >> caching)
> > >> >> > to trunk. Colin McCabe and I have been hard at work the last 3.5
> > >> months
> > >> >> > implementing this feature, and feel that it's reached a level of
> > >> >> stability
> > >> >> > and utility where it's ready for broader testing and integration.
> > >> >> >
> > >> >> > I'd also like to thank Chris Nauroth at Hortonworks for code
> > reviews
> > >> and
> > >> >> > bug fixes, and everyone who's reviewed the HDFS-4949 design doc
> and
> > >> left
> > >> >> > comments.
> > >> >> >
> > >> >> > Obviously, I am +1 for the merge. The vote will run the standard
> 7
> > >> days,
> > >> >> > closing on October 24 at 11:59PM.
> > >> >> >
> > >> >> > Thanks,
> > >> >> > Andrew
> > >> >>
> > >> >
> > >> > --
> > >> > CONFIDENTIALITY NOTICE
> > >> > NOTICE: This message is intended for the use of the individual or
> > >> entity to
> > >> > which it is addressed and may contain information that is
> > confidential,
> > >> > privileged and exempt from disclosure under applicable law. If the
> > >> reader
> > >> > of this message is not the intended recipient, you are hereby
> notified
> > >> that
> > >> > any printing, copying, dissemination, distribution, disclosure or
> > >> > forwarding of this communication is strictly prohibited. If you have
> > >> > received this communication in error, please contact the sender
> > >> immediately
> > >> > and delete it from your system. Thank You.
> > >>
> > >
> > >
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: [VOTE] Merge HDFS-4949 to trunk

[Andrew Wang <an...@cloudera.com>]

Hey Chris,

Right now we're on track to have all of those things done by tomorrow.
Since the remaining issues are either not technical or do not involve major
changes, I was hoping we could +1 this merge vote in the spirit of "+1
pending jenkins". We've gotten clean unit test runs on upstream Jenkins as
well, so the only fixups we should need for test-patch.sh are findbugs and
javac (which are normally pretty trivial to clean up). Of course, all of
your listed prereqs and test-patch would be taken care of before actually
merging to trunk.

So, we can reset the vote if you feel strongly about this, but it seems
like the only real result will be delaying the merge by a week.

Thanks,
Andrew


On Wed, Oct 23, 2013 at 1:03 PM, Chris Nauroth <cn...@hortonworks.com>wrote:

> I've received some feedback that we haven't handled this merge vote the
> same as other comparable merge votes, and that the vote should be reset
> because of this.
>
> The recent custom is that we only call for the merge vote after all
> pre-requisites have been satisfied.  This would include committing to the
> feature branch all patches that the devs deem necessary before the code
> lands in trunk, posting a test plan, posting an updated design doc in case
> implementation choices diverged from the original design doc, and getting a
> good test-patch run from Jenkins on the merge patch.  This was the process
> followed for other recent major features like HDFS-2802 (snapshots),
> HDFS-347 (short-circuit reads via sharing file descriptors), and
> HADOOP-8562 (Windows compatibility).  In this thread, we've diverged from
> that process by calling for a vote on a branch that hasn't yet completed
> the pre-requisites and stating a plan for work to be done before the merge.
>
> I still support this work, but can we please restart the vote after the
> pre-requisites have landed in the branch?
>
> Chris Nauroth
> Hortonworks
> http://hortonworks.com/
>
>
>
> On Fri, Oct 18, 2013 at 1:37 PM, Chris Nauroth <cnauroth@hortonworks.com
> >wrote:
>
> > +1
> >
> > Sounds great!
> >
> > Regarding testing caching+federation, this is another thing that I had
> > intended to pick up as part of HDFS-5149.  I'm not sure if I can get this
> > done in the next 7 days, so I'll keep you posted.
> >
> > Chris Nauroth
> > Hortonworks
> > http://hortonworks.com/
> >
> >
> >
> > On Fri, Oct 18, 2013 at 11:15 AM, Colin McCabe <cmccabe@alumni.cmu.edu
> >wrote:
> >
> >> Hi Chris,
> >>
> >> I think it's feasible to complete those tasks in the next 7 days.
> >> Andrew is on HDFS-5386.
> >>
> >> The test plan document is a great idea.  We'll try to get that up
> >> early next week.  We have a lot of unit tests now, clearly, but some
> >> manual testing is important too.
> >>
> >> If we discover any issues during testing, then we can push out the
> >> merge timeframe.  For example, one area that probably needs more
> >> testing is caching+federation.
> >>
> >> I would like to get HDFS-5378 and HDFS-5366 in as well.
> >>
> >> The other subtasks are "nice to have" but not really critical, and I
> >> think it would be just as easy to do them in trunk.  We're hoping that
> >> having this in trunk will make it easier for us to collaborate on
> >> HDFS-2832 and other ongoing work.
> >>
> >> > Also, I want to confirm that this vote only covers trunk.
> >> > I don't see branch-2 mentioned, so I assume that we're
> >> > not voting on merge to branch-2 yet.
> >>
> >> Yeah, this vote is only to merge to trunk.
> >>
> >> cheers.
> >> Colin
> >>
> >> On Fri, Oct 18, 2013 at 10:48 AM, Chris Nauroth
> >> <cn...@hortonworks.com> wrote:
> >> > I agree that the code has reached a stable point.  Colin and Andrew,
> >> thank
> >> > you for your contributions and collaboration.
> >> >
> >> > Throughout development, I've watched the feature grow by running daily
> >> > builds in a pseudo-distributed deployment.  As of this week, the full
> >> > feature set is working end-to-end.  I also think we've reached a point
> >> of
> >> > API stability for clients who want to control caching
> programmatically.
> >> >
> >> > There are several things that I'd like to see completed before the
> >> merge as
> >> > pre-requisites:
> >> >
> >> > - HDFS-5203: Concurrent clients that add a cache directive on the same
> >> path
> >> > may prematurely uncache from each other.
> >> > - HDFS-5385: Caching RPCs are AtMostOnce, but do not persist client ID
> >> and
> >> > call ID to edit log.
> >> > - HDFS-5386: Add feature documentation for datanode caching.
> >> > - Standard clean-ups to satisfy Jenkins pre-commit on the merge patch.
> >> >  (For example, I know we've introduced some Javadoc warnings.)
> >> > - Full test suite run on Windows.  (The feature is not yet implemented
> >> on
> >> > Windows.  This is just intended to catch regressions.)
> >> > - Test plan posted to HDFS-4949, similar in scope to the snapshot test
> >> plan
> >> > that was posted to HDFS-2802.  For my own part, I've run the new unit
> >> > tests, and I've tested end-to-end in a pseudo-distributed deployment.
> >>  It's
> >> > unlikely that I'll get a chance to test fully distributed before the
> >> vote
> >> > closes, so I'm curious to hear if you've done this on your side yet.
> >> >
> >> > Also, I want to confirm that this vote only covers trunk.  I don't see
> >> > branch-2 mentioned, so I assume that we're not voting on merge to
> >> branch-2
> >> > yet.
> >> >
> >> > Before I cast my vote, can you please discuss whether or not it's
> >> feasible
> >> > to complete all of the above in the next 7 days?  For the issues
> >> assigned
> >> > to me, I do expect to complete them.
> >> >
> >> > Thanks again for all of your hard work!
> >> >
> >> > Chris Nauroth
> >> > Hortonworks
> >> > http://hortonworks.com/
> >> >
> >> >
> >> >
> >> > On Thu, Oct 17, 2013 at 3:07 PM, Colin McCabe <cmccabe@alumni.cmu.edu
> >> >wrote:
> >> >
> >> >> +1.  Thanks, guys.
> >> >>
> >> >> best,
> >> >> Colin
> >> >>
> >> >> On Thu, Oct 17, 2013 at 3:01 PM, Andrew Wang <
> andrew.wang@cloudera.com
> >> >
> >> >> wrote:
> >> >> > Hello all,
> >> >> >
> >> >> > I'd like to call a vote to merge the HDFS-4949 branch (in-memory
> >> caching)
> >> >> > to trunk. Colin McCabe and I have been hard at work the last 3.5
> >> months
> >> >> > implementing this feature, and feel that it's reached a level of
> >> >> stability
> >> >> > and utility where it's ready for broader testing and integration.
> >> >> >
> >> >> > I'd also like to thank Chris Nauroth at Hortonworks for code
> reviews
> >> and
> >> >> > bug fixes, and everyone who's reviewed the HDFS-4949 design doc and
> >> left
> >> >> > comments.
> >> >> >
> >> >> > Obviously, I am +1 for the merge. The vote will run the standard 7
> >> days,
> >> >> > closing on October 24 at 11:59PM.
> >> >> >
> >> >> > Thanks,
> >> >> > Andrew
> >> >>
> >> >
> >> > --
> >> > CONFIDENTIALITY NOTICE
> >> > NOTICE: This message is intended for the use of the individual or
> >> entity to
> >> > which it is addressed and may contain information that is
> confidential,
> >> > privileged and exempt from disclosure under applicable law. If the
> >> reader
> >> > of this message is not the intended recipient, you are hereby notified
> >> that
> >> > any printing, copying, dissemination, distribution, disclosure or
> >> > forwarding of this communication is strictly prohibited. If you have
> >> > received this communication in error, please contact the sender
> >> immediately
> >> > and delete it from your system. Thank You.
> >>
> >
> >
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Re: [VOTE] Merge HDFS-4949 to trunk

[Andrew Wang <an...@cloudera.com>]

Hey Chris,

Right now we're on track to have all of those things done by tomorrow.
Since the remaining issues are either not technical or do not involve major
changes, I was hoping we could +1 this merge vote in the spirit of "+1
pending jenkins". We've gotten clean unit test runs on upstream Jenkins as
well, so the only fixups we should need for test-patch.sh are findbugs and
javac (which are normally pretty trivial to clean up). Of course, all of
your listed prereqs and test-patch would be taken care of before actually
merging to trunk.

So, we can reset the vote if you feel strongly about this, but it seems
like the only real result will be delaying the merge by a week.

Thanks,
Andrew


On Wed, Oct 23, 2013 at 1:03 PM, Chris Nauroth <cn...@hortonworks.com>wrote:

> I've received some feedback that we haven't handled this merge vote the
> same as other comparable merge votes, and that the vote should be reset
> because of this.
>
> The recent custom is that we only call for the merge vote after all
> pre-requisites have been satisfied.  This would include committing to the
> feature branch all patches that the devs deem necessary before the code
> lands in trunk, posting a test plan, posting an updated design doc in case
> implementation choices diverged from the original design doc, and getting a
> good test-patch run from Jenkins on the merge patch.  This was the process
> followed for other recent major features like HDFS-2802 (snapshots),
> HDFS-347 (short-circuit reads via sharing file descriptors), and
> HADOOP-8562 (Windows compatibility).  In this thread, we've diverged from
> that process by calling for a vote on a branch that hasn't yet completed
> the pre-requisites and stating a plan for work to be done before the merge.
>
> I still support this work, but can we please restart the vote after the
> pre-requisites have landed in the branch?
>
> Chris Nauroth
> Hortonworks
> http://hortonworks.com/
>
>
>
> On Fri, Oct 18, 2013 at 1:37 PM, Chris Nauroth <cnauroth@hortonworks.com
> >wrote:
>
> > +1
> >
> > Sounds great!
> >
> > Regarding testing caching+federation, this is another thing that I had
> > intended to pick up as part of HDFS-5149.  I'm not sure if I can get this
> > done in the next 7 days, so I'll keep you posted.
> >
> > Chris Nauroth
> > Hortonworks
> > http://hortonworks.com/
> >
> >
> >
> > On Fri, Oct 18, 2013 at 11:15 AM, Colin McCabe <cmccabe@alumni.cmu.edu
> >wrote:
> >
> >> Hi Chris,
> >>
> >> I think it's feasible to complete those tasks in the next 7 days.
> >> Andrew is on HDFS-5386.
> >>
> >> The test plan document is a great idea.  We'll try to get that up
> >> early next week.  We have a lot of unit tests now, clearly, but some
> >> manual testing is important too.
> >>
> >> If we discover any issues during testing, then we can push out the
> >> merge timeframe.  For example, one area that probably needs more
> >> testing is caching+federation.
> >>
> >> I would like to get HDFS-5378 and HDFS-5366 in as well.
> >>
> >> The other subtasks are "nice to have" but not really critical, and I
> >> think it would be just as easy to do them in trunk.  We're hoping that
> >> having this in trunk will make it easier for us to collaborate on
> >> HDFS-2832 and other ongoing work.
> >>
> >> > Also, I want to confirm that this vote only covers trunk.
> >> > I don't see branch-2 mentioned, so I assume that we're
> >> > not voting on merge to branch-2 yet.
> >>
> >> Yeah, this vote is only to merge to trunk.
> >>
> >> cheers.
> >> Colin
> >>
> >> On Fri, Oct 18, 2013 at 10:48 AM, Chris Nauroth
> >> <cn...@hortonworks.com> wrote:
> >> > I agree that the code has reached a stable point.  Colin and Andrew,
> >> thank
> >> > you for your contributions and collaboration.
> >> >
> >> > Throughout development, I've watched the feature grow by running daily
> >> > builds in a pseudo-distributed deployment.  As of this week, the full
> >> > feature set is working end-to-end.  I also think we've reached a point
> >> of
> >> > API stability for clients who want to control caching
> programmatically.
> >> >
> >> > There are several things that I'd like to see completed before the
> >> merge as
> >> > pre-requisites:
> >> >
> >> > - HDFS-5203: Concurrent clients that add a cache directive on the same
> >> path
> >> > may prematurely uncache from each other.
> >> > - HDFS-5385: Caching RPCs are AtMostOnce, but do not persist client ID
> >> and
> >> > call ID to edit log.
> >> > - HDFS-5386: Add feature documentation for datanode caching.
> >> > - Standard clean-ups to satisfy Jenkins pre-commit on the merge patch.
> >> >  (For example, I know we've introduced some Javadoc warnings.)
> >> > - Full test suite run on Windows.  (The feature is not yet implemented
> >> on
> >> > Windows.  This is just intended to catch regressions.)
> >> > - Test plan posted to HDFS-4949, similar in scope to the snapshot test
> >> plan
> >> > that was posted to HDFS-2802.  For my own part, I've run the new unit
> >> > tests, and I've tested end-to-end in a pseudo-distributed deployment.
> >>  It's
> >> > unlikely that I'll get a chance to test fully distributed before the
> >> vote
> >> > closes, so I'm curious to hear if you've done this on your side yet.
> >> >
> >> > Also, I want to confirm that this vote only covers trunk.  I don't see
> >> > branch-2 mentioned, so I assume that we're not voting on merge to
> >> branch-2
> >> > yet.
> >> >
> >> > Before I cast my vote, can you please discuss whether or not it's
> >> feasible
> >> > to complete all of the above in the next 7 days?  For the issues
> >> assigned
> >> > to me, I do expect to complete them.
> >> >
> >> > Thanks again for all of your hard work!
> >> >
> >> > Chris Nauroth
> >> > Hortonworks
> >> > http://hortonworks.com/
> >> >
> >> >
> >> >
> >> > On Thu, Oct 17, 2013 at 3:07 PM, Colin McCabe <cmccabe@alumni.cmu.edu
> >> >wrote:
> >> >
> >> >> +1.  Thanks, guys.
> >> >>
> >> >> best,
> >> >> Colin
> >> >>
> >> >> On Thu, Oct 17, 2013 at 3:01 PM, Andrew Wang <
> andrew.wang@cloudera.com
> >> >
> >> >> wrote:
> >> >> > Hello all,
> >> >> >
> >> >> > I'd like to call a vote to merge the HDFS-4949 branch (in-memory
> >> caching)
> >> >> > to trunk. Colin McCabe and I have been hard at work the last 3.5
> >> months
> >> >> > implementing this feature, and feel that it's reached a level of
> >> >> stability
> >> >> > and utility where it's ready for broader testing and integration.
> >> >> >
> >> >> > I'd also like to thank Chris Nauroth at Hortonworks for code
> reviews
> >> and
> >> >> > bug fixes, and everyone who's reviewed the HDFS-4949 design doc and
> >> left
> >> >> > comments.
> >> >> >
> >> >> > Obviously, I am +1 for the merge. The vote will run the standard 7
> >> days,
> >> >> > closing on October 24 at 11:59PM.
> >> >> >
> >> >> > Thanks,
> >> >> > Andrew
> >> >>
> >> >
> >> > --
> >> > CONFIDENTIALITY NOTICE
> >> > NOTICE: This message is intended for the use of the individual or
> >> entity to
> >> > which it is addressed and may contain information that is
> confidential,
> >> > privileged and exempt from disclosure under applicable law. If the
> >> reader
> >> > of this message is not the intended recipient, you are hereby notified
> >> that
> >> > any printing, copying, dissemination, distribution, disclosure or
> >> > forwarding of this communication is strictly prohibited. If you have
> >> > received this communication in error, please contact the sender
> >> immediately
> >> > and delete it from your system. Thank You.
> >>
> >
> >
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Re: [VOTE] Merge HDFS-4949 to trunk

[Chris Nauroth <cn...@hortonworks.com>]

I've received some feedback that we haven't handled this merge vote the
same as other comparable merge votes, and that the vote should be reset
because of this.

The recent custom is that we only call for the merge vote after all
pre-requisites have been satisfied.  This would include committing to the
feature branch all patches that the devs deem necessary before the code
lands in trunk, posting a test plan, posting an updated design doc in case
implementation choices diverged from the original design doc, and getting a
good test-patch run from Jenkins on the merge patch.  This was the process
followed for other recent major features like HDFS-2802 (snapshots),
HDFS-347 (short-circuit reads via sharing file descriptors), and
HADOOP-8562 (Windows compatibility).  In this thread, we've diverged from
that process by calling for a vote on a branch that hasn't yet completed
the pre-requisites and stating a plan for work to be done before the merge.

I still support this work, but can we please restart the vote after the
pre-requisites have landed in the branch?

Chris Nauroth
Hortonworks
http://hortonworks.com/



On Fri, Oct 18, 2013 at 1:37 PM, Chris Nauroth <cn...@hortonworks.com>wrote:

> +1
>
> Sounds great!
>
> Regarding testing caching+federation, this is another thing that I had
> intended to pick up as part of HDFS-5149.  I'm not sure if I can get this
> done in the next 7 days, so I'll keep you posted.
>
> Chris Nauroth
> Hortonworks
> http://hortonworks.com/
>
>
>
> On Fri, Oct 18, 2013 at 11:15 AM, Colin McCabe <cm...@alumni.cmu.edu>wrote:
>
>> Hi Chris,
>>
>> I think it's feasible to complete those tasks in the next 7 days.
>> Andrew is on HDFS-5386.
>>
>> The test plan document is a great idea.  We'll try to get that up
>> early next week.  We have a lot of unit tests now, clearly, but some
>> manual testing is important too.
>>
>> If we discover any issues during testing, then we can push out the
>> merge timeframe.  For example, one area that probably needs more
>> testing is caching+federation.
>>
>> I would like to get HDFS-5378 and HDFS-5366 in as well.
>>
>> The other subtasks are "nice to have" but not really critical, and I
>> think it would be just as easy to do them in trunk.  We're hoping that
>> having this in trunk will make it easier for us to collaborate on
>> HDFS-2832 and other ongoing work.
>>
>> > Also, I want to confirm that this vote only covers trunk.
>> > I don't see branch-2 mentioned, so I assume that we're
>> > not voting on merge to branch-2 yet.
>>
>> Yeah, this vote is only to merge to trunk.
>>
>> cheers.
>> Colin
>>
>> On Fri, Oct 18, 2013 at 10:48 AM, Chris Nauroth
>> <cn...@hortonworks.com> wrote:
>> > I agree that the code has reached a stable point.  Colin and Andrew,
>> thank
>> > you for your contributions and collaboration.
>> >
>> > Throughout development, I've watched the feature grow by running daily
>> > builds in a pseudo-distributed deployment.  As of this week, the full
>> > feature set is working end-to-end.  I also think we've reached a point
>> of
>> > API stability for clients who want to control caching programmatically.
>> >
>> > There are several things that I'd like to see completed before the
>> merge as
>> > pre-requisites:
>> >
>> > - HDFS-5203: Concurrent clients that add a cache directive on the same
>> path
>> > may prematurely uncache from each other.
>> > - HDFS-5385: Caching RPCs are AtMostOnce, but do not persist client ID
>> and
>> > call ID to edit log.
>> > - HDFS-5386: Add feature documentation for datanode caching.
>> > - Standard clean-ups to satisfy Jenkins pre-commit on the merge patch.
>> >  (For example, I know we've introduced some Javadoc warnings.)
>> > - Full test suite run on Windows.  (The feature is not yet implemented
>> on
>> > Windows.  This is just intended to catch regressions.)
>> > - Test plan posted to HDFS-4949, similar in scope to the snapshot test
>> plan
>> > that was posted to HDFS-2802.  For my own part, I've run the new unit
>> > tests, and I've tested end-to-end in a pseudo-distributed deployment.
>>  It's
>> > unlikely that I'll get a chance to test fully distributed before the
>> vote
>> > closes, so I'm curious to hear if you've done this on your side yet.
>> >
>> > Also, I want to confirm that this vote only covers trunk.  I don't see
>> > branch-2 mentioned, so I assume that we're not voting on merge to
>> branch-2
>> > yet.
>> >
>> > Before I cast my vote, can you please discuss whether or not it's
>> feasible
>> > to complete all of the above in the next 7 days?  For the issues
>> assigned
>> > to me, I do expect to complete them.
>> >
>> > Thanks again for all of your hard work!
>> >
>> > Chris Nauroth
>> > Hortonworks
>> > http://hortonworks.com/
>> >
>> >
>> >
>> > On Thu, Oct 17, 2013 at 3:07 PM, Colin McCabe <cmccabe@alumni.cmu.edu
>> >wrote:
>> >
>> >> +1.  Thanks, guys.
>> >>
>> >> best,
>> >> Colin
>> >>
>> >> On Thu, Oct 17, 2013 at 3:01 PM, Andrew Wang <andrew.wang@cloudera.com
>> >
>> >> wrote:
>> >> > Hello all,
>> >> >
>> >> > I'd like to call a vote to merge the HDFS-4949 branch (in-memory
>> caching)
>> >> > to trunk. Colin McCabe and I have been hard at work the last 3.5
>> months
>> >> > implementing this feature, and feel that it's reached a level of
>> >> stability
>> >> > and utility where it's ready for broader testing and integration.
>> >> >
>> >> > I'd also like to thank Chris Nauroth at Hortonworks for code reviews
>> and
>> >> > bug fixes, and everyone who's reviewed the HDFS-4949 design doc and
>> left
>> >> > comments.
>> >> >
>> >> > Obviously, I am +1 for the merge. The vote will run the standard 7
>> days,
>> >> > closing on October 24 at 11:59PM.
>> >> >
>> >> > Thanks,
>> >> > Andrew
>> >>
>> >
>> > --
>> > CONFIDENTIALITY NOTICE
>> > NOTICE: This message is intended for the use of the individual or
>> entity to
>> > which it is addressed and may contain information that is confidential,
>> > privileged and exempt from disclosure under applicable law. If the
>> reader
>> > of this message is not the intended recipient, you are hereby notified
>> that
>> > any printing, copying, dissemination, distribution, disclosure or
>> > forwarding of this communication is strictly prohibited. If you have
>> > received this communication in error, please contact the sender
>> immediately
>> > and delete it from your system. Thank You.
>>
>
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: [VOTE] Merge HDFS-4949 to trunk

[Chris Nauroth <cn...@hortonworks.com>]

I've received some feedback that we haven't handled this merge vote the
same as other comparable merge votes, and that the vote should be reset
because of this.

The recent custom is that we only call for the merge vote after all
pre-requisites have been satisfied.  This would include committing to the
feature branch all patches that the devs deem necessary before the code
lands in trunk, posting a test plan, posting an updated design doc in case
implementation choices diverged from the original design doc, and getting a
good test-patch run from Jenkins on the merge patch.  This was the process
followed for other recent major features like HDFS-2802 (snapshots),
HDFS-347 (short-circuit reads via sharing file descriptors), and
HADOOP-8562 (Windows compatibility).  In this thread, we've diverged from
that process by calling for a vote on a branch that hasn't yet completed
the pre-requisites and stating a plan for work to be done before the merge.

I still support this work, but can we please restart the vote after the
pre-requisites have landed in the branch?

Chris Nauroth
Hortonworks
http://hortonworks.com/



On Fri, Oct 18, 2013 at 1:37 PM, Chris Nauroth <cn...@hortonworks.com>wrote:

> +1
>
> Sounds great!
>
> Regarding testing caching+federation, this is another thing that I had
> intended to pick up as part of HDFS-5149.  I'm not sure if I can get this
> done in the next 7 days, so I'll keep you posted.
>
> Chris Nauroth
> Hortonworks
> http://hortonworks.com/
>
>
>
> On Fri, Oct 18, 2013 at 11:15 AM, Colin McCabe <cm...@alumni.cmu.edu>wrote:
>
>> Hi Chris,
>>
>> I think it's feasible to complete those tasks in the next 7 days.
>> Andrew is on HDFS-5386.
>>
>> The test plan document is a great idea.  We'll try to get that up
>> early next week.  We have a lot of unit tests now, clearly, but some
>> manual testing is important too.
>>
>> If we discover any issues during testing, then we can push out the
>> merge timeframe.  For example, one area that probably needs more
>> testing is caching+federation.
>>
>> I would like to get HDFS-5378 and HDFS-5366 in as well.
>>
>> The other subtasks are "nice to have" but not really critical, and I
>> think it would be just as easy to do them in trunk.  We're hoping that
>> having this in trunk will make it easier for us to collaborate on
>> HDFS-2832 and other ongoing work.
>>
>> > Also, I want to confirm that this vote only covers trunk.
>> > I don't see branch-2 mentioned, so I assume that we're
>> > not voting on merge to branch-2 yet.
>>
>> Yeah, this vote is only to merge to trunk.
>>
>> cheers.
>> Colin
>>
>> On Fri, Oct 18, 2013 at 10:48 AM, Chris Nauroth
>> <cn...@hortonworks.com> wrote:
>> > I agree that the code has reached a stable point.  Colin and Andrew,
>> thank
>> > you for your contributions and collaboration.
>> >
>> > Throughout development, I've watched the feature grow by running daily
>> > builds in a pseudo-distributed deployment.  As of this week, the full
>> > feature set is working end-to-end.  I also think we've reached a point
>> of
>> > API stability for clients who want to control caching programmatically.
>> >
>> > There are several things that I'd like to see completed before the
>> merge as
>> > pre-requisites:
>> >
>> > - HDFS-5203: Concurrent clients that add a cache directive on the same
>> path
>> > may prematurely uncache from each other.
>> > - HDFS-5385: Caching RPCs are AtMostOnce, but do not persist client ID
>> and
>> > call ID to edit log.
>> > - HDFS-5386: Add feature documentation for datanode caching.
>> > - Standard clean-ups to satisfy Jenkins pre-commit on the merge patch.
>> >  (For example, I know we've introduced some Javadoc warnings.)
>> > - Full test suite run on Windows.  (The feature is not yet implemented
>> on
>> > Windows.  This is just intended to catch regressions.)
>> > - Test plan posted to HDFS-4949, similar in scope to the snapshot test
>> plan
>> > that was posted to HDFS-2802.  For my own part, I've run the new unit
>> > tests, and I've tested end-to-end in a pseudo-distributed deployment.
>>  It's
>> > unlikely that I'll get a chance to test fully distributed before the
>> vote
>> > closes, so I'm curious to hear if you've done this on your side yet.
>> >
>> > Also, I want to confirm that this vote only covers trunk.  I don't see
>> > branch-2 mentioned, so I assume that we're not voting on merge to
>> branch-2
>> > yet.
>> >
>> > Before I cast my vote, can you please discuss whether or not it's
>> feasible
>> > to complete all of the above in the next 7 days?  For the issues
>> assigned
>> > to me, I do expect to complete them.
>> >
>> > Thanks again for all of your hard work!
>> >
>> > Chris Nauroth
>> > Hortonworks
>> > http://hortonworks.com/
>> >
>> >
>> >
>> > On Thu, Oct 17, 2013 at 3:07 PM, Colin McCabe <cmccabe@alumni.cmu.edu
>> >wrote:
>> >
>> >> +1.  Thanks, guys.
>> >>
>> >> best,
>> >> Colin
>> >>
>> >> On Thu, Oct 17, 2013 at 3:01 PM, Andrew Wang <andrew.wang@cloudera.com
>> >
>> >> wrote:
>> >> > Hello all,
>> >> >
>> >> > I'd like to call a vote to merge the HDFS-4949 branch (in-memory
>> caching)
>> >> > to trunk. Colin McCabe and I have been hard at work the last 3.5
>> months
>> >> > implementing this feature, and feel that it's reached a level of
>> >> stability
>> >> > and utility where it's ready for broader testing and integration.
>> >> >
>> >> > I'd also like to thank Chris Nauroth at Hortonworks for code reviews
>> and
>> >> > bug fixes, and everyone who's reviewed the HDFS-4949 design doc and
>> left
>> >> > comments.
>> >> >
>> >> > Obviously, I am +1 for the merge. The vote will run the standard 7
>> days,
>> >> > closing on October 24 at 11:59PM.
>> >> >
>> >> > Thanks,
>> >> > Andrew
>> >>
>> >
>> > --
>> > CONFIDENTIALITY NOTICE
>> > NOTICE: This message is intended for the use of the individual or
>> entity to
>> > which it is addressed and may contain information that is confidential,
>> > privileged and exempt from disclosure under applicable law. If the
>> reader
>> > of this message is not the intended recipient, you are hereby notified
>> that
>> > any printing, copying, dissemination, distribution, disclosure or
>> > forwarding of this communication is strictly prohibited. If you have
>> > received this communication in error, please contact the sender
>> immediately
>> > and delete it from your system. Thank You.
>>
>
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: [VOTE] Merge HDFS-4949 to trunk

[Chris Nauroth <cn...@hortonworks.com>]

+1

Sounds great!

Regarding testing caching+federation, this is another thing that I had
intended to pick up as part of HDFS-5149.  I'm not sure if I can get this
done in the next 7 days, so I'll keep you posted.

Chris Nauroth
Hortonworks
http://hortonworks.com/



On Fri, Oct 18, 2013 at 11:15 AM, Colin McCabe <cm...@alumni.cmu.edu>wrote:

> Hi Chris,
>
> I think it's feasible to complete those tasks in the next 7 days.
> Andrew is on HDFS-5386.
>
> The test plan document is a great idea.  We'll try to get that up
> early next week.  We have a lot of unit tests now, clearly, but some
> manual testing is important too.
>
> If we discover any issues during testing, then we can push out the
> merge timeframe.  For example, one area that probably needs more
> testing is caching+federation.
>
> I would like to get HDFS-5378 and HDFS-5366 in as well.
>
> The other subtasks are "nice to have" but not really critical, and I
> think it would be just as easy to do them in trunk.  We're hoping that
> having this in trunk will make it easier for us to collaborate on
> HDFS-2832 and other ongoing work.
>
> > Also, I want to confirm that this vote only covers trunk.
> > I don't see branch-2 mentioned, so I assume that we're
> > not voting on merge to branch-2 yet.
>
> Yeah, this vote is only to merge to trunk.
>
> cheers.
> Colin
>
> On Fri, Oct 18, 2013 at 10:48 AM, Chris Nauroth
> <cn...@hortonworks.com> wrote:
> > I agree that the code has reached a stable point.  Colin and Andrew,
> thank
> > you for your contributions and collaboration.
> >
> > Throughout development, I've watched the feature grow by running daily
> > builds in a pseudo-distributed deployment.  As of this week, the full
> > feature set is working end-to-end.  I also think we've reached a point of
> > API stability for clients who want to control caching programmatically.
> >
> > There are several things that I'd like to see completed before the merge
> as
> > pre-requisites:
> >
> > - HDFS-5203: Concurrent clients that add a cache directive on the same
> path
> > may prematurely uncache from each other.
> > - HDFS-5385: Caching RPCs are AtMostOnce, but do not persist client ID
> and
> > call ID to edit log.
> > - HDFS-5386: Add feature documentation for datanode caching.
> > - Standard clean-ups to satisfy Jenkins pre-commit on the merge patch.
> >  (For example, I know we've introduced some Javadoc warnings.)
> > - Full test suite run on Windows.  (The feature is not yet implemented on
> > Windows.  This is just intended to catch regressions.)
> > - Test plan posted to HDFS-4949, similar in scope to the snapshot test
> plan
> > that was posted to HDFS-2802.  For my own part, I've run the new unit
> > tests, and I've tested end-to-end in a pseudo-distributed deployment.
>  It's
> > unlikely that I'll get a chance to test fully distributed before the vote
> > closes, so I'm curious to hear if you've done this on your side yet.
> >
> > Also, I want to confirm that this vote only covers trunk.  I don't see
> > branch-2 mentioned, so I assume that we're not voting on merge to
> branch-2
> > yet.
> >
> > Before I cast my vote, can you please discuss whether or not it's
> feasible
> > to complete all of the above in the next 7 days?  For the issues assigned
> > to me, I do expect to complete them.
> >
> > Thanks again for all of your hard work!
> >
> > Chris Nauroth
> > Hortonworks
> > http://hortonworks.com/
> >
> >
> >
> > On Thu, Oct 17, 2013 at 3:07 PM, Colin McCabe <cmccabe@alumni.cmu.edu
> >wrote:
> >
> >> +1.  Thanks, guys.
> >>
> >> best,
> >> Colin
> >>
> >> On Thu, Oct 17, 2013 at 3:01 PM, Andrew Wang <an...@cloudera.com>
> >> wrote:
> >> > Hello all,
> >> >
> >> > I'd like to call a vote to merge the HDFS-4949 branch (in-memory
> caching)
> >> > to trunk. Colin McCabe and I have been hard at work the last 3.5
> months
> >> > implementing this feature, and feel that it's reached a level of
> >> stability
> >> > and utility where it's ready for broader testing and integration.
> >> >
> >> > I'd also like to thank Chris Nauroth at Hortonworks for code reviews
> and
> >> > bug fixes, and everyone who's reviewed the HDFS-4949 design doc and
> left
> >> > comments.
> >> >
> >> > Obviously, I am +1 for the merge. The vote will run the standard 7
> days,
> >> > closing on October 24 at 11:59PM.
> >> >
> >> > Thanks,
> >> > Andrew
> >>
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: [VOTE] Merge HDFS-4949 to trunk

[Chris Nauroth <cn...@hortonworks.com>]

+1

Sounds great!

Regarding testing caching+federation, this is another thing that I had
intended to pick up as part of HDFS-5149.  I'm not sure if I can get this
done in the next 7 days, so I'll keep you posted.

Chris Nauroth
Hortonworks
http://hortonworks.com/



On Fri, Oct 18, 2013 at 11:15 AM, Colin McCabe <cm...@alumni.cmu.edu>wrote:

> Hi Chris,
>
> I think it's feasible to complete those tasks in the next 7 days.
> Andrew is on HDFS-5386.
>
> The test plan document is a great idea.  We'll try to get that up
> early next week.  We have a lot of unit tests now, clearly, but some
> manual testing is important too.
>
> If we discover any issues during testing, then we can push out the
> merge timeframe.  For example, one area that probably needs more
> testing is caching+federation.
>
> I would like to get HDFS-5378 and HDFS-5366 in as well.
>
> The other subtasks are "nice to have" but not really critical, and I
> think it would be just as easy to do them in trunk.  We're hoping that
> having this in trunk will make it easier for us to collaborate on
> HDFS-2832 and other ongoing work.
>
> > Also, I want to confirm that this vote only covers trunk.
> > I don't see branch-2 mentioned, so I assume that we're
> > not voting on merge to branch-2 yet.
>
> Yeah, this vote is only to merge to trunk.
>
> cheers.
> Colin
>
> On Fri, Oct 18, 2013 at 10:48 AM, Chris Nauroth
> <cn...@hortonworks.com> wrote:
> > I agree that the code has reached a stable point.  Colin and Andrew,
> thank
> > you for your contributions and collaboration.
> >
> > Throughout development, I've watched the feature grow by running daily
> > builds in a pseudo-distributed deployment.  As of this week, the full
> > feature set is working end-to-end.  I also think we've reached a point of
> > API stability for clients who want to control caching programmatically.
> >
> > There are several things that I'd like to see completed before the merge
> as
> > pre-requisites:
> >
> > - HDFS-5203: Concurrent clients that add a cache directive on the same
> path
> > may prematurely uncache from each other.
> > - HDFS-5385: Caching RPCs are AtMostOnce, but do not persist client ID
> and
> > call ID to edit log.
> > - HDFS-5386: Add feature documentation for datanode caching.
> > - Standard clean-ups to satisfy Jenkins pre-commit on the merge patch.
> >  (For example, I know we've introduced some Javadoc warnings.)
> > - Full test suite run on Windows.  (The feature is not yet implemented on
> > Windows.  This is just intended to catch regressions.)
> > - Test plan posted to HDFS-4949, similar in scope to the snapshot test
> plan
> > that was posted to HDFS-2802.  For my own part, I've run the new unit
> > tests, and I've tested end-to-end in a pseudo-distributed deployment.
>  It's
> > unlikely that I'll get a chance to test fully distributed before the vote
> > closes, so I'm curious to hear if you've done this on your side yet.
> >
> > Also, I want to confirm that this vote only covers trunk.  I don't see
> > branch-2 mentioned, so I assume that we're not voting on merge to
> branch-2
> > yet.
> >
> > Before I cast my vote, can you please discuss whether or not it's
> feasible
> > to complete all of the above in the next 7 days?  For the issues assigned
> > to me, I do expect to complete them.
> >
> > Thanks again for all of your hard work!
> >
> > Chris Nauroth
> > Hortonworks
> > http://hortonworks.com/
> >
> >
> >
> > On Thu, Oct 17, 2013 at 3:07 PM, Colin McCabe <cmccabe@alumni.cmu.edu
> >wrote:
> >
> >> +1.  Thanks, guys.
> >>
> >> best,
> >> Colin
> >>
> >> On Thu, Oct 17, 2013 at 3:01 PM, Andrew Wang <an...@cloudera.com>
> >> wrote:
> >> > Hello all,
> >> >
> >> > I'd like to call a vote to merge the HDFS-4949 branch (in-memory
> caching)
> >> > to trunk. Colin McCabe and I have been hard at work the last 3.5
> months
> >> > implementing this feature, and feel that it's reached a level of
> >> stability
> >> > and utility where it's ready for broader testing and integration.
> >> >
> >> > I'd also like to thank Chris Nauroth at Hortonworks for code reviews
> and
> >> > bug fixes, and everyone who's reviewed the HDFS-4949 design doc and
> left
> >> > comments.
> >> >
> >> > Obviously, I am +1 for the merge. The vote will run the standard 7
> days,
> >> > closing on October 24 at 11:59PM.
> >> >
> >> > Thanks,
> >> > Andrew
> >>
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: [VOTE] Merge HDFS-4949 to trunk

[Colin McCabe <cm...@alumni.cmu.edu>]

Hi Chris,

I think it's feasible to complete those tasks in the next 7 days.
Andrew is on HDFS-5386.

The test plan document is a great idea.  We'll try to get that up
early next week.  We have a lot of unit tests now, clearly, but some
manual testing is important too.

If we discover any issues during testing, then we can push out the
merge timeframe.  For example, one area that probably needs more
testing is caching+federation.

I would like to get HDFS-5378 and HDFS-5366 in as well.

The other subtasks are "nice to have" but not really critical, and I
think it would be just as easy to do them in trunk.  We're hoping that
having this in trunk will make it easier for us to collaborate on
HDFS-2832 and other ongoing work.

> Also, I want to confirm that this vote only covers trunk.
> I don't see branch-2 mentioned, so I assume that we're
> not voting on merge to branch-2 yet.

Yeah, this vote is only to merge to trunk.

cheers.
Colin

On Fri, Oct 18, 2013 at 10:48 AM, Chris Nauroth
<cn...@hortonworks.com> wrote:
> I agree that the code has reached a stable point.  Colin and Andrew, thank
> you for your contributions and collaboration.
>
> Throughout development, I've watched the feature grow by running daily
> builds in a pseudo-distributed deployment.  As of this week, the full
> feature set is working end-to-end.  I also think we've reached a point of
> API stability for clients who want to control caching programmatically.
>
> There are several things that I'd like to see completed before the merge as
> pre-requisites:
>
> - HDFS-5203: Concurrent clients that add a cache directive on the same path
> may prematurely uncache from each other.
> - HDFS-5385: Caching RPCs are AtMostOnce, but do not persist client ID and
> call ID to edit log.
> - HDFS-5386: Add feature documentation for datanode caching.
> - Standard clean-ups to satisfy Jenkins pre-commit on the merge patch.
>  (For example, I know we've introduced some Javadoc warnings.)
> - Full test suite run on Windows.  (The feature is not yet implemented on
> Windows.  This is just intended to catch regressions.)
> - Test plan posted to HDFS-4949, similar in scope to the snapshot test plan
> that was posted to HDFS-2802.  For my own part, I've run the new unit
> tests, and I've tested end-to-end in a pseudo-distributed deployment.  It's
> unlikely that I'll get a chance to test fully distributed before the vote
> closes, so I'm curious to hear if you've done this on your side yet.
>
> Also, I want to confirm that this vote only covers trunk.  I don't see
> branch-2 mentioned, so I assume that we're not voting on merge to branch-2
> yet.
>
> Before I cast my vote, can you please discuss whether or not it's feasible
> to complete all of the above in the next 7 days?  For the issues assigned
> to me, I do expect to complete them.
>
> Thanks again for all of your hard work!
>
> Chris Nauroth
> Hortonworks
> http://hortonworks.com/
>
>
>
> On Thu, Oct 17, 2013 at 3:07 PM, Colin McCabe <cm...@alumni.cmu.edu>wrote:
>
>> +1.  Thanks, guys.
>>
>> best,
>> Colin
>>
>> On Thu, Oct 17, 2013 at 3:01 PM, Andrew Wang <an...@cloudera.com>
>> wrote:
>> > Hello all,
>> >
>> > I'd like to call a vote to merge the HDFS-4949 branch (in-memory caching)
>> > to trunk. Colin McCabe and I have been hard at work the last 3.5 months
>> > implementing this feature, and feel that it's reached a level of
>> stability
>> > and utility where it's ready for broader testing and integration.
>> >
>> > I'd also like to thank Chris Nauroth at Hortonworks for code reviews and
>> > bug fixes, and everyone who's reviewed the HDFS-4949 design doc and left
>> > comments.
>> >
>> > Obviously, I am +1 for the merge. The vote will run the standard 7 days,
>> > closing on October 24 at 11:59PM.
>> >
>> > Thanks,
>> > Andrew
>>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.

Re: [VOTE] Merge HDFS-4949 to trunk

[Colin McCabe <cm...@alumni.cmu.edu>]

Hi Chris,

I think it's feasible to complete those tasks in the next 7 days.
Andrew is on HDFS-5386.

The test plan document is a great idea.  We'll try to get that up
early next week.  We have a lot of unit tests now, clearly, but some
manual testing is important too.

If we discover any issues during testing, then we can push out the
merge timeframe.  For example, one area that probably needs more
testing is caching+federation.

I would like to get HDFS-5378 and HDFS-5366 in as well.

The other subtasks are "nice to have" but not really critical, and I
think it would be just as easy to do them in trunk.  We're hoping that
having this in trunk will make it easier for us to collaborate on
HDFS-2832 and other ongoing work.

> Also, I want to confirm that this vote only covers trunk.
> I don't see branch-2 mentioned, so I assume that we're
> not voting on merge to branch-2 yet.

Yeah, this vote is only to merge to trunk.

cheers.
Colin

On Fri, Oct 18, 2013 at 10:48 AM, Chris Nauroth
<cn...@hortonworks.com> wrote:
> I agree that the code has reached a stable point.  Colin and Andrew, thank
> you for your contributions and collaboration.
>
> Throughout development, I've watched the feature grow by running daily
> builds in a pseudo-distributed deployment.  As of this week, the full
> feature set is working end-to-end.  I also think we've reached a point of
> API stability for clients who want to control caching programmatically.
>
> There are several things that I'd like to see completed before the merge as
> pre-requisites:
>
> - HDFS-5203: Concurrent clients that add a cache directive on the same path
> may prematurely uncache from each other.
> - HDFS-5385: Caching RPCs are AtMostOnce, but do not persist client ID and
> call ID to edit log.
> - HDFS-5386: Add feature documentation for datanode caching.
> - Standard clean-ups to satisfy Jenkins pre-commit on the merge patch.
>  (For example, I know we've introduced some Javadoc warnings.)
> - Full test suite run on Windows.  (The feature is not yet implemented on
> Windows.  This is just intended to catch regressions.)
> - Test plan posted to HDFS-4949, similar in scope to the snapshot test plan
> that was posted to HDFS-2802.  For my own part, I've run the new unit
> tests, and I've tested end-to-end in a pseudo-distributed deployment.  It's
> unlikely that I'll get a chance to test fully distributed before the vote
> closes, so I'm curious to hear if you've done this on your side yet.
>
> Also, I want to confirm that this vote only covers trunk.  I don't see
> branch-2 mentioned, so I assume that we're not voting on merge to branch-2
> yet.
>
> Before I cast my vote, can you please discuss whether or not it's feasible
> to complete all of the above in the next 7 days?  For the issues assigned
> to me, I do expect to complete them.
>
> Thanks again for all of your hard work!
>
> Chris Nauroth
> Hortonworks
> http://hortonworks.com/
>
>
>
> On Thu, Oct 17, 2013 at 3:07 PM, Colin McCabe <cm...@alumni.cmu.edu>wrote:
>
>> +1.  Thanks, guys.
>>
>> best,
>> Colin
>>
>> On Thu, Oct 17, 2013 at 3:01 PM, Andrew Wang <an...@cloudera.com>
>> wrote:
>> > Hello all,
>> >
>> > I'd like to call a vote to merge the HDFS-4949 branch (in-memory caching)
>> > to trunk. Colin McCabe and I have been hard at work the last 3.5 months
>> > implementing this feature, and feel that it's reached a level of
>> stability
>> > and utility where it's ready for broader testing and integration.
>> >
>> > I'd also like to thank Chris Nauroth at Hortonworks for code reviews and
>> > bug fixes, and everyone who's reviewed the HDFS-4949 design doc and left
>> > comments.
>> >
>> > Obviously, I am +1 for the merge. The vote will run the standard 7 days,
>> > closing on October 24 at 11:59PM.
>> >
>> > Thanks,
>> > Andrew
>>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.

Re: [VOTE] Merge HDFS-4949 to trunk

[Chris Nauroth <cn...@hortonworks.com>]

I agree that the code has reached a stable point.  Colin and Andrew, thank
you for your contributions and collaboration.

Throughout development, I've watched the feature grow by running daily
builds in a pseudo-distributed deployment.  As of this week, the full
feature set is working end-to-end.  I also think we've reached a point of
API stability for clients who want to control caching programmatically.

There are several things that I'd like to see completed before the merge as
pre-requisites:

- HDFS-5203: Concurrent clients that add a cache directive on the same path
may prematurely uncache from each other.
- HDFS-5385: Caching RPCs are AtMostOnce, but do not persist client ID and
call ID to edit log.
- HDFS-5386: Add feature documentation for datanode caching.
- Standard clean-ups to satisfy Jenkins pre-commit on the merge patch.
 (For example, I know we've introduced some Javadoc warnings.)
- Full test suite run on Windows.  (The feature is not yet implemented on
Windows.  This is just intended to catch regressions.)
- Test plan posted to HDFS-4949, similar in scope to the snapshot test plan
that was posted to HDFS-2802.  For my own part, I've run the new unit
tests, and I've tested end-to-end in a pseudo-distributed deployment.  It's
unlikely that I'll get a chance to test fully distributed before the vote
closes, so I'm curious to hear if you've done this on your side yet.

Also, I want to confirm that this vote only covers trunk.  I don't see
branch-2 mentioned, so I assume that we're not voting on merge to branch-2
yet.

Before I cast my vote, can you please discuss whether or not it's feasible
to complete all of the above in the next 7 days?  For the issues assigned
to me, I do expect to complete them.

Thanks again for all of your hard work!

Chris Nauroth
Hortonworks
http://hortonworks.com/



On Thu, Oct 17, 2013 at 3:07 PM, Colin McCabe <cm...@alumni.cmu.edu>wrote:

> +1.  Thanks, guys.
>
> best,
> Colin
>
> On Thu, Oct 17, 2013 at 3:01 PM, Andrew Wang <an...@cloudera.com>
> wrote:
> > Hello all,
> >
> > I'd like to call a vote to merge the HDFS-4949 branch (in-memory caching)
> > to trunk. Colin McCabe and I have been hard at work the last 3.5 months
> > implementing this feature, and feel that it's reached a level of
> stability
> > and utility where it's ready for broader testing and integration.
> >
> > I'd also like to thank Chris Nauroth at Hortonworks for code reviews and
> > bug fixes, and everyone who's reviewed the HDFS-4949 design doc and left
> > comments.
> >
> > Obviously, I am +1 for the merge. The vote will run the standard 7 days,
> > closing on October 24 at 11:59PM.
> >
> > Thanks,
> > Andrew
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: [VOTE] Merge HDFS-4949 to trunk

[Chris Nauroth <cn...@hortonworks.com>]

I agree that the code has reached a stable point.  Colin and Andrew, thank
you for your contributions and collaboration.

Throughout development, I've watched the feature grow by running daily
builds in a pseudo-distributed deployment.  As of this week, the full
feature set is working end-to-end.  I also think we've reached a point of
API stability for clients who want to control caching programmatically.

There are several things that I'd like to see completed before the merge as
pre-requisites:

- HDFS-5203: Concurrent clients that add a cache directive on the same path
may prematurely uncache from each other.
- HDFS-5385: Caching RPCs are AtMostOnce, but do not persist client ID and
call ID to edit log.
- HDFS-5386: Add feature documentation for datanode caching.
- Standard clean-ups to satisfy Jenkins pre-commit on the merge patch.
 (For example, I know we've introduced some Javadoc warnings.)
- Full test suite run on Windows.  (The feature is not yet implemented on
Windows.  This is just intended to catch regressions.)
- Test plan posted to HDFS-4949, similar in scope to the snapshot test plan
that was posted to HDFS-2802.  For my own part, I've run the new unit
tests, and I've tested end-to-end in a pseudo-distributed deployment.  It's
unlikely that I'll get a chance to test fully distributed before the vote
closes, so I'm curious to hear if you've done this on your side yet.

Also, I want to confirm that this vote only covers trunk.  I don't see
branch-2 mentioned, so I assume that we're not voting on merge to branch-2
yet.

Before I cast my vote, can you please discuss whether or not it's feasible
to complete all of the above in the next 7 days?  For the issues assigned
to me, I do expect to complete them.

Thanks again for all of your hard work!

Chris Nauroth
Hortonworks
http://hortonworks.com/



On Thu, Oct 17, 2013 at 3:07 PM, Colin McCabe <cm...@alumni.cmu.edu>wrote:

> +1.  Thanks, guys.
>
> best,
> Colin
>
> On Thu, Oct 17, 2013 at 3:01 PM, Andrew Wang <an...@cloudera.com>
> wrote:
> > Hello all,
> >
> > I'd like to call a vote to merge the HDFS-4949 branch (in-memory caching)
> > to trunk. Colin McCabe and I have been hard at work the last 3.5 months
> > implementing this feature, and feel that it's reached a level of
> stability
> > and utility where it's ready for broader testing and integration.
> >
> > I'd also like to thank Chris Nauroth at Hortonworks for code reviews and
> > bug fixes, and everyone who's reviewed the HDFS-4949 design doc and left
> > comments.
> >
> > Obviously, I am +1 for the merge. The vote will run the standard 7 days,
> > closing on October 24 at 11:59PM.
> >
> > Thanks,
> > Andrew
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: [VOTE] Merge HDFS-4949 to trunk

[Colin McCabe <cm...@alumni.cmu.edu>]

+1.  Thanks, guys.

best,
Colin

On Thu, Oct 17, 2013 at 3:01 PM, Andrew Wang <an...@cloudera.com> wrote:
> Hello all,
>
> I'd like to call a vote to merge the HDFS-4949 branch (in-memory caching)
> to trunk. Colin McCabe and I have been hard at work the last 3.5 months
> implementing this feature, and feel that it's reached a level of stability
> and utility where it's ready for broader testing and integration.
>
> I'd also like to thank Chris Nauroth at Hortonworks for code reviews and
> bug fixes, and everyone who's reviewed the HDFS-4949 design doc and left
> comments.
>
> Obviously, I am +1 for the merge. The vote will run the standard 7 days,
> closing on October 24 at 11:59PM.
>
> Thanks,
> Andrew

Re: [VOTE] Merge HDFS-4949 to trunk

[Colin McCabe <cm...@alumni.cmu.edu>]

+1.  Thanks, guys.

best,
Colin

On Thu, Oct 17, 2013 at 3:01 PM, Andrew Wang <an...@cloudera.com> wrote:
> Hello all,
>
> I'd like to call a vote to merge the HDFS-4949 branch (in-memory caching)
> to trunk. Colin McCabe and I have been hard at work the last 3.5 months
> implementing this feature, and feel that it's reached a level of stability
> and utility where it's ready for broader testing and integration.
>
> I'd also like to thank Chris Nauroth at Hortonworks for code reviews and
> bug fixes, and everyone who's reviewed the HDFS-4949 design doc and left
> comments.
>
> Obviously, I am +1 for the merge. The vote will run the standard 7 days,
> closing on October 24 at 11:59PM.
>
> Thanks,
> Andrew