Re: Crunch build system refactoring

[Josh Wills <jw...@cloudera.com>]

Hi Matthias,

Thanks for reaching out. Your timing was perfect, as I had a similar
thought this morning before I woke up and checked my email. I'm so glad
that someone else was already on top of it.

As an FYI, Crunch has moved to the Apache incubator, and we're in the
process of porting over all of the infrastructure associated with the
project. I created a JIRA to track the Maven refactoring here:
https://issues.apache.org/jira/browse/CRUNCH-8

If you would like to submit your change as a patch that we could submit to
the project, we would be grateful. We're always on the lookout for new
committers, and this would be an excellent contribution to the project.

The new origin for the project is here:

http://git-wip-us.apache.org/repos/asf/incubator-crunch.git

We'll get all of the traditional Apache/Github stuff up in the next couple
of weeks, which should help make this a bit easier. If you need any help in
the meantime, please do not hesitate to reach out to the
crunch-dev@incubator.apache.org mailing list.

Best,
Josh

On Sun, Jul 8, 2012 at 1:57 AM, Matthias Friedrich <
matthias.j.friedrich@googlemail.com> wrote:

> Hi,
>
> I recently discovered crunch and it looks very promising. We use plain MR
> jobs heavily at work but I really like the idea of raising the abstraction
> level a bit while still working with Java and staying flexible. So crunch
> could be a long-term option for us.
>
> Anyway, I read that you're planning to use continuous integration. This
> means you'll probably need to refactor the build system a bit to make sure
> all of your code (examples and scrunch) is built along with crunch itself.
> The usual Maven way of doing this is to turn crunch into a multi-module
> project. I put the changes to do this on a branch in case you're
> interested: https://github.com/mafr/crunch/tree/maven-multi-module
>
> This is a rather large change with lots of XML juggling, so please make
> sure it's to your liking before merging it. Build systems are never quite
> done, but I'd lend a hand if you need any help.
>
> Regards,
>   Matthias
>



-- 
Director of Data Science
Cloudera <http://www.cloudera.com>
Twitter: @josh_wills <http://twitter.com/josh_wills>