Re: Native task branch progress

[Todd Lipcon <to...@apache.org>]

Hi folks,

Just to follow up here: benchmark and cluster validation results are
available on the MR-2841 JIRA. Over the next few days we're closing out a
couple small remaining issues and I'll start posting candidate merge
patches for Jenkins to test.

I anticipate calling the merge vote in parallel with the Jenkins precommit
checks and any small follow-ons that that might expose. If there's any
specific testing items you feel you need to see to support this merge,
please let me know now.

Thanks
-Todd


On Tue, Aug 26, 2014 at 11:50 AM, Todd Lipcon <to...@apache.org> wrote:

> Hi all,
>
> I wanted to give a quick status update on the MAPREDUCE-2841 feature
> branch which was started earlier this summer.
>
> Since establishing the branch, we've accomplished a number of key items:
>
> - Imported the initial code from github
> - Made integrations with other projects more pluggable using a
> ServiceLoader mechanism
> - Integrated the project build into our normal maven structure and release
> tarballs
> - Fixed several unit test failures that occurred in different developer
> environments
> - Fixed a couple of licensing issues with code
> - Various code cleanup for style, etc.
> - Fixed all of the compiler warnings
>
> I've also spent some time doing comparative microbenchmarks of the mapside
> sort between this implementation, the existing MR implementation, and
> Facebook's implementation (available on github). I haven't posted those
> yet, but I hope to do so later this week. The summary is that the native
> implementation compares quite favorably to the other options.
>
> This week, I'm hoping to close out a couple remaining items:
>
> - A few small code cleanup items I have pending locally (eg improving
> javadocs, etc)
> - Some basic cluster verification and performance tests
> - Post benchmark results mentioned above
>
> After these items are done, I think we are ready to merge this feature to
> trunk. After it is in trunk, we can continue to make improvements, add
> documentation, etc. Because the patch is structured as completely new code
> (no changes to existing functionality, manually enabled on a per-job
> basis), this should be a "zero-risk" merge.
>
> I'm anticipating calling this merge vote next week, and would appreciate
> if other committers can plan to be ready to vote at that time. That is to
> say, if you anticipate wanting to review the branch merge, this is your
> "heads up" that the vote is coming. Please start your reviews this week.
>
> Thanks
> -Todd
>



-- 
Todd Lipcon
Software Engineer, Cloudera