You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Nick Pentreath <ni...@gmail.com> on 2013/12/02 18:09:59 UTC
PySpark / scikit-learn integration sprint at Cloudera - Strata
Conference Friday 14th Feb 2014
Hi Spark Devs
An idea developed recently out of a scikit-learn mailing list discussion (
http://sourceforge.net/mailarchive/forum.php?thread_name=CAFvE7K5HGKYH9Myp7imrJ-nU%3DpJgeGqcCn3JC0m4MmGWZi35Hw%40mail.gmail.com&forum_name=scikit-learn-general)
to have a coding sprint around Strata in Feb, focused on integration
between scikit-learn and PySpark for large-scale machine learning tasks.
Cloudera has kindly agreed to host the sprint, most likely in San
Francisco. Ideally it would be focused and capped at around 10 people. The
idea is not meant to be a teaching workshop for
newcomers but more as a prototyping session, so ideally it would be great
to have developers and users with deep knowledge of PySpark (Josh
especially :) and/or scikit-learn, attend.
Hopefully we can get some people from the Spark community involved, and
Olivier will drum up support from the scikit-learn community.
All the best and hope to see you there (though likely I will only be able
to join remotely).
Nick
Re: PySpark / scikit-learn integration sprint at Cloudera - Strata
Conference Friday 14th Feb 2014
Posted by Olivier Grisel <ol...@ensta.org>.
2013/12/4 Josh Rosen <ro...@gmail.com>:
> Thanks for organizing this! I'll definitely be attending.
Great. Looking forward to meet you to.
Uri, you might want to register as well on the wiki :)
--
Olivier
http://twitter.com/ogrisel - http://github.com/ogrisel
Re: PySpark / scikit-learn integration sprint at Cloudera - Strata
Conference Friday 14th Feb 2014
Posted by Josh Rosen <ro...@gmail.com>.
Thanks for organizing this! I'll definitely be attending.
- Josh
On Wed, Dec 4, 2013 at 6:07 AM, Olivier Grisel <ol...@ensta.org>wrote:
> 2013/12/4 Olivier Grisel <ol...@ensta.org>:
> > That should be fixed but only if nobody clicks on the previous URL...
> > Use the following instead:
> >
> > https://github.com/scikit-learn/scikit-learn/wiki/Upcoming-events
> >
> > That's a weird github bug...
>
> I switched the rendering to markdown and the page is stable now. Sorry for
> that.
>
> --
> Olivier
> http://twitter.com/ogrisel - http://github.com/ogrisel
>
Re: PySpark / scikit-learn integration sprint at Cloudera - Strata
Conference Friday 14th Feb 2014
Posted by Olivier Grisel <ol...@ensta.org>.
2013/12/4 Olivier Grisel <ol...@ensta.org>:
> That should be fixed but only if nobody clicks on the previous URL...
> Use the following instead:
>
> https://github.com/scikit-learn/scikit-learn/wiki/Upcoming-events
>
> That's a weird github bug...
I switched the rendering to markdown and the page is stable now. Sorry for that.
--
Olivier
http://twitter.com/ogrisel - http://github.com/ogrisel
Re: PySpark / scikit-learn integration sprint at Cloudera - Strata
Conference Friday 14th Feb 2014
Posted by Olivier Grisel <ol...@ensta.org>.
That should be fixed but only if nobody clicks on the previous URL...
Use the following instead:
https://github.com/scikit-learn/scikit-learn/wiki/Upcoming-events
That's a weird github bug...
--
Olivier
Re: PySpark / scikit-learn integration sprint at Cloudera - Strata
Conference Friday 14th Feb 2014
Posted by Olivier Grisel <ol...@ensta.org>.
Oops. I got something wrong with the markup. Let me fix it first.
--
Olivier
Re: PySpark / scikit-learn integration sprint at Cloudera - Strata
Conference Friday 14th Feb 2014
Posted by Olivier Grisel <ol...@ensta.org>.
2013/12/3 Horia <ho...@alum.berkeley.edu>:
> I am very interested in this and will most definitely participate!
>
> Please share the event sign-up list and location details when all the
> organizational hurdles have been resolved :-)
Great! I just created a new entry for this sprint on the scikit-learn wiki:
https://github.com/scikit-learn/scikit-learn/wiki/Upcoming-events#scikit-learn--pyspark-integration-sprint---friday-14-february-2014
Please feel free to register there.
--
Olivier
http://twitter.com/ogrisel - http://github.com/ogrisel
Re: PySpark / scikit-learn integration sprint at Cloudera - Strata
Conference Friday 14th Feb 2014
Posted by Olivier Grisel <ol...@ensta.org>.
Done.
--
Olivier
Re: PySpark / scikit-learn integration sprint at Cloudera - Strata
Conference Friday 14th Feb 2014
Posted by Justin Kestelyn <jk...@cloudera.com>.
Please also add to the venue info: The group will be on the 6th floor.
On Wed, Dec 11, 2013 at 8:14 AM, Horia <ho...@alum.berkeley.edu> wrote:
> If it's up for a vote, I'd say we start at 9:30 :)
>
>
>
> On Wed, Dec 11, 2013 at 1:56 AM, Olivier Grisel <ol...@ensta.org>wrote:
>
>> 2013/12/10 Justin Kestelyn <jk...@cloudera.com>:
>> > Location:
>> >
>> > Cloudera
>> > 433 California Street
>> > San Francisco, CA
>>
>> Thanks I will add that on the wiki, shall we start at 9am? 9.30am?
>>
>> --
>> Olivier
>>
>
>
--
*Justin Kestelyn*
Developer Outreach & Relations
Cloudera | www.cloudera.com
Tel: 650.683.4688
Email: jkestelyn@cloudera.com
Twitter: @kestelyn, @ClouderaEng
Re: PySpark / scikit-learn integration sprint at Cloudera - Strata
Conference Friday 14th Feb 2014
Posted by Horia <ho...@alum.berkeley.edu>.
If it's up for a vote, I'd say we start at 9:30 :)
On Wed, Dec 11, 2013 at 1:56 AM, Olivier Grisel <ol...@ensta.org>wrote:
> 2013/12/10 Justin Kestelyn <jk...@cloudera.com>:
> > Location:
> >
> > Cloudera
> > 433 California Street
> > San Francisco, CA
>
> Thanks I will add that on the wiki, shall we start at 9am? 9.30am?
>
> --
> Olivier
>
Re: PySpark / scikit-learn integration sprint at Cloudera - Strata
Conference Friday 14th Feb 2014
Posted by Olivier Grisel <ol...@ensta.org>.
2013/12/10 Justin Kestelyn <jk...@cloudera.com>:
> Location:
>
> Cloudera
> 433 California Street
> San Francisco, CA
Thanks I will add that on the wiki, shall we start at 9am? 9.30am?
--
Olivier
Re: PySpark / scikit-learn integration sprint at Cloudera - Strata
Conference Friday 14th Feb 2014
Posted by Justin Kestelyn <jk...@cloudera.com>.
Location:
Cloudera
433 California Street
San Francisco, CA
On Tue, Dec 3, 2013 at 9:30 AM, Horia <ho...@alum.berkeley.edu> wrote:
> I am very interested in this and will most definitely participate!
>
> Please share the event sign-up list and location details when all the
> organizational hurdles have been resolved :-)
>
>
>
> On Mon, Dec 2, 2013 at 1:45 PM, Reynold Xin <rx...@apache.org> wrote:
>
>> Definitely some people will get confused. It's up to you. If we post it,
>> we
>> can mark it in the title that this is a hackathon.
>>
>>
>> On Mon, Dec 2, 2013 at 1:43 PM, Olivier Grisel <olivier.grisel@ensta.org
>> >wrote:
>>
>> > 2013/12/2 Reynold Xin <rx...@apache.org>:
>> > > Including the link to the meetup group:
>> > http://www.meetup.com/spark-users/
>> >
>> > I am not opposed to it but I am wondering if people will not confuse
>> > it with a traditional meetup if we do so.
>> >
>> > --
>> > Olivier
>> >
>>
>
>
--
*Justin Kestelyn*
Developer Outreach
Cloudera | www.cloudera.com
Tel: 650.683.4688
Email: jkestelyn@cloudera.com
Twitter: @kestelyn, @ClouderaEng
Re: PySpark / scikit-learn integration sprint at Cloudera - Strata
Conference Friday 14th Feb 2014
Posted by Horia <ho...@alum.berkeley.edu>.
I am very interested in this and will most definitely participate!
Please share the event sign-up list and location details when all the
organizational hurdles have been resolved :-)
On Mon, Dec 2, 2013 at 1:45 PM, Reynold Xin <rx...@apache.org> wrote:
> Definitely some people will get confused. It's up to you. If we post it, we
> can mark it in the title that this is a hackathon.
>
>
> On Mon, Dec 2, 2013 at 1:43 PM, Olivier Grisel <olivier.grisel@ensta.org
> >wrote:
>
> > 2013/12/2 Reynold Xin <rx...@apache.org>:
> > > Including the link to the meetup group:
> > http://www.meetup.com/spark-users/
> >
> > I am not opposed to it but I am wondering if people will not confuse
> > it with a traditional meetup if we do so.
> >
> > --
> > Olivier
> >
>
Re: PySpark / scikit-learn integration sprint at Cloudera - Strata
Conference Friday 14th Feb 2014
Posted by Reynold Xin <rx...@apache.org>.
Definitely some people will get confused. It's up to you. If we post it, we
can mark it in the title that this is a hackathon.
On Mon, Dec 2, 2013 at 1:43 PM, Olivier Grisel <ol...@ensta.org>wrote:
> 2013/12/2 Reynold Xin <rx...@apache.org>:
> > Including the link to the meetup group:
> http://www.meetup.com/spark-users/
>
> I am not opposed to it but I am wondering if people will not confuse
> it with a traditional meetup if we do so.
>
> --
> Olivier
>
Re: PySpark / scikit-learn integration sprint at Cloudera - Strata
Conference Friday 14th Feb 2014
Posted by Olivier Grisel <ol...@ensta.org>.
2013/12/2 Reynold Xin <rx...@apache.org>:
> Including the link to the meetup group: http://www.meetup.com/spark-users/
I am not opposed to it but I am wondering if people will not confuse
it with a traditional meetup if we do so.
--
Olivier
Re: PySpark / scikit-learn integration sprint at Cloudera - Strata
Conference Friday 14th Feb 2014
Posted by Reynold Xin <rx...@apache.org>.
Including the link to the meetup group: http://www.meetup.com/spark-users/
On Mon, Dec 2, 2013 at 1:22 PM, Reynold Xin <rx...@apache.org> wrote:
> Olivier,
>
> Do you want us to create a Spark user meetup event for this hackathon?
>
> On Mon, Dec 2, 2013 at 1:12 PM, Olivier Grisel <ol...@ensta.org>wrote:
>
>> Hi all,
>>
>> Just a quick reply to say that I would be glad to meet some of you to
>> hack on some prototype scikit-learn / PySpark integration.
>>
>> Cloudera just confirmed that we have a room for us at their San
>> Fransisco offices on Friday Feb 14 (right after Strata).
>>
>> Hope to see you there or at Strata,
>>
>> --
>> Olivier
>>
>
>
Re: PySpark / scikit-learn integration sprint at Cloudera - Strata
Conference Friday 14th Feb 2014
Posted by Reynold Xin <rx...@apache.org>.
Olivier,
Do you want us to create a Spark user meetup event for this hackathon?
On Mon, Dec 2, 2013 at 1:12 PM, Olivier Grisel <ol...@ensta.org>wrote:
> Hi all,
>
> Just a quick reply to say that I would be glad to meet some of you to
> hack on some prototype scikit-learn / PySpark integration.
>
> Cloudera just confirmed that we have a room for us at their San
> Fransisco offices on Friday Feb 14 (right after Strata).
>
> Hope to see you there or at Strata,
>
> --
> Olivier
>
Re: PySpark / scikit-learn integration sprint at Cloudera - Strata
Conference Friday 14th Feb 2014
Posted by Olivier Grisel <ol...@ensta.org>.
Hi all,
Just a quick reply to say that I would be glad to meet some of you to
hack on some prototype scikit-learn / PySpark integration.
Cloudera just confirmed that we have a room for us at their San
Fransisco offices on Friday Feb 14 (right after Strata).
Hope to see you there or at Strata,
--
Olivier