You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@celeborn.apache.org by Ethan Feng <et...@apache.org> on 2023/09/14 11:43:25 UTC

[NEW FEATURE AVAILABLE] Celeborn support MapReduce engine.

Hello developers and users,
  I am glad to announce that Celeborn supports the MapReduce engine
now.  Both Hadoop 2 and 3 are supported. If you are interested, you
can just try it and feedback on anything you want.


  The quick start guide can be found here [
https://celeborn.apache.org/docs/latest/ ].
  The design doc can be found here [
https://docs.google.com/document/d/1g4irlBucIAFNI42cFSuOVWYqOWSvuqpw_VBHmyyv8zo/edit?usp=sharing
].

Thanks,
Ethan.

Re: [NEW FEATURE AVAILABLE] Celeborn support MapReduce engine.

Posted by Lewis John McGibbney <le...@apache.org>.
Excellent. Thanks for the heads up :) 
lewismc

On 2023/10/18 03:44:54 Ethan Feng wrote:
> Hi Lewis,
> 
> Thanks for reaching out.
> 
> I can confirm that future Celeborn releases will include the "mr"
> client jars since Celeborn 0.4.0 and it will start the release process
> in a short period.
> 
> If you have any further questions or concerns about using MapReduce
> with Celeborn, please don't hesitate to let me know.
> 
> Best regards,
> Ethan Feng

Re: [NEW FEATURE AVAILABLE] Celeborn support MapReduce engine.

Posted by Ethan Feng <et...@apache.org>.
Hi Lewis,

Thanks for reaching out.

I can confirm that future Celeborn releases will include the "mr"
client jars since Celeborn 0.4.0 and it will start the release process
in a short period.

If you have any further questions or concerns about using MapReduce
with Celeborn, please don't hesitate to let me know.

Best regards,
Ethan Feng

Lewis John McGibbney <le...@apache.org> 于2023年10月18日周三 11:12写道：
>
> Hi Ethan,
>
> I'm just picking up Celeborn now and plan on running some experiments with the Apache Nutch (https://nutch.apache.org) project.
>
> I downloaded Celeborn 0.3.1-incubating (2023-10-13) from the downloads page and noticed that no Celeborn client jars for MapReduce exist at $CELEBORN_HOME/mr/*.jar as suggested within the documentation at https://celeborn.apache.org/docs/latest/#add-celeborn-client-jar-to-mapreduces-classpath
>
> I'm cloned the source (69defca) right now and built with ./build/make-distribution.sh -Pmr, I now see the 'mr' directory which I can use...
>
> Out of curiosity will future Celeborn releases include the "mr" client jars?
>
> Thank you
> lewismc
>
> On 2023/09/14 11:43:25 Ethan Feng wrote:
> > Hello developers and users,
> >   I am glad to announce that Celeborn supports the MapReduce engine
> > now.  Both Hadoop 2 and 3 are supported. If you are interested, you
> > can just try it and feedback on anything you want.
> >
> >
> >   The quick start guide can be found here [
> > https://celeborn.apache.org/docs/latest/ ].
> >   The design doc can be found here [
> > https://docs.google.com/document/d/1g4irlBucIAFNI42cFSuOVWYqOWSvuqpw_VBHmyyv8zo/edit?usp=sharing
> > ].
> >
> > Thanks,
> > Ethan.
> >

Re: [NEW FEATURE AVAILABLE] Celeborn support MapReduce engine.

Posted by Lewis John McGibbney <le...@apache.org>.
Hi Ethan,

I'm just picking up Celeborn now and plan on running some experiments with the Apache Nutch (https://nutch.apache.org) project.

I downloaded Celeborn 0.3.1-incubating (2023-10-13) from the downloads page and noticed that no Celeborn client jars for MapReduce exist at $CELEBORN_HOME/mr/*.jar as suggested within the documentation at https://celeborn.apache.org/docs/latest/#add-celeborn-client-jar-to-mapreduces-classpath

I'm cloned the source (69defca) right now and built with ./build/make-distribution.sh -Pmr, I now see the 'mr' directory which I can use...

Out of curiosity will future Celeborn releases include the "mr" client jars?

Thank you
lewismc

On 2023/09/14 11:43:25 Ethan Feng wrote:
> Hello developers and users,
>   I am glad to announce that Celeborn supports the MapReduce engine
> now.  Both Hadoop 2 and 3 are supported. If you are interested, you
> can just try it and feedback on anything you want.
> 
> 
>   The quick start guide can be found here [
> https://celeborn.apache.org/docs/latest/ ].
>   The design doc can be found here [
> https://docs.google.com/document/d/1g4irlBucIAFNI42cFSuOVWYqOWSvuqpw_VBHmyyv8zo/edit?usp=sharing
> ].
> 
> Thanks,
> Ethan.
>