You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Nick Pentreath <ni...@gmail.com> on 2013/12/02 18:09:59 UTC

PySpark / scikit-learn integration sprint at Cloudera - Strata Conference Friday 14th Feb 2014

Hi Spark Devs

An idea developed recently out of a scikit-learn mailing list discussion (
http://sourceforge.net/mailarchive/forum.php?thread_name=CAFvE7K5HGKYH9Myp7imrJ-nU%3DpJgeGqcCn3JC0m4MmGWZi35Hw%40mail.gmail.com&forum_name=scikit-learn-general)
to have a coding sprint around Strata in Feb, focused on integration
between scikit-learn and PySpark for large-scale machine learning tasks.

Cloudera has kindly agreed to host the sprint, most likely in San
Francisco. Ideally it would be focused and capped at around 10 people. The
idea is not meant to be a teaching workshop for
newcomers but more as a prototyping session, so ideally it would be great
to have developers and users with deep knowledge of PySpark (Josh
especially :) and/or scikit-learn, attend.

Hopefully we can get some people from the Spark community involved, and
Olivier will drum up support from the scikit-learn community.

All the best and hope to see you there (though likely I will only be able
to join remotely).
Nick

Re: PySpark / scikit-learn integration sprint at Cloudera - Strata Conference Friday 14th Feb 2014

Posted by Olivier Grisel <ol...@ensta.org>.
2013/12/4 Josh Rosen <ro...@gmail.com>:
> Thanks for organizing this!  I'll definitely be attending.

Great. Looking forward to meet you to.

Uri, you might want to register as well on the wiki :)

-- 
Olivier
http://twitter.com/ogrisel - http://github.com/ogrisel

Re: PySpark / scikit-learn integration sprint at Cloudera - Strata Conference Friday 14th Feb 2014

Posted by Josh Rosen <ro...@gmail.com>.
Thanks for organizing this!  I'll definitely be attending.

- Josh


On Wed, Dec 4, 2013 at 6:07 AM, Olivier Grisel <ol...@ensta.org>wrote:

> 2013/12/4 Olivier Grisel <ol...@ensta.org>:
> > That should be fixed but only if nobody clicks on the previous URL...
> > Use the following instead:
> >
> > https://github.com/scikit-learn/scikit-learn/wiki/Upcoming-events
> >
> > That's a weird github bug...
>
> I switched the rendering to markdown and the page is stable now. Sorry for
> that.
>
> --
> Olivier
> http://twitter.com/ogrisel - http://github.com/ogrisel
>

Re: PySpark / scikit-learn integration sprint at Cloudera - Strata Conference Friday 14th Feb 2014

Posted by Olivier Grisel <ol...@ensta.org>.
2013/12/4 Olivier Grisel <ol...@ensta.org>:
> That should be fixed but only if nobody clicks on the previous URL...
> Use the following instead:
>
> https://github.com/scikit-learn/scikit-learn/wiki/Upcoming-events
>
> That's a weird github bug...

I switched the rendering to markdown and the page is stable now. Sorry for that.

-- 
Olivier
http://twitter.com/ogrisel - http://github.com/ogrisel

Re: PySpark / scikit-learn integration sprint at Cloudera - Strata Conference Friday 14th Feb 2014

Posted by Olivier Grisel <ol...@ensta.org>.
That should be fixed but only if nobody clicks on the previous URL...
Use the following instead:

https://github.com/scikit-learn/scikit-learn/wiki/Upcoming-events

That's a weird github bug...

-- 
Olivier

Re: PySpark / scikit-learn integration sprint at Cloudera - Strata Conference Friday 14th Feb 2014

Posted by Olivier Grisel <ol...@ensta.org>.
Oops. I got something wrong with the markup. Let me fix it first.

-- 
Olivier

Re: PySpark / scikit-learn integration sprint at Cloudera - Strata Conference Friday 14th Feb 2014

Posted by Olivier Grisel <ol...@ensta.org>.
2013/12/3 Horia <ho...@alum.berkeley.edu>:
> I am very interested in this and will most definitely participate!
>
> Please share the event sign-up list and location details when all the
> organizational hurdles have been resolved :-)

Great! I just created a new entry for this sprint on the scikit-learn wiki:

  https://github.com/scikit-learn/scikit-learn/wiki/Upcoming-events#scikit-learn--pyspark-integration-sprint---friday-14-february-2014

Please feel free to register there.

-- 
Olivier
http://twitter.com/ogrisel - http://github.com/ogrisel

Re: PySpark / scikit-learn integration sprint at Cloudera - Strata Conference Friday 14th Feb 2014

Posted by Olivier Grisel <ol...@ensta.org>.
Done.

-- 
Olivier

Re: PySpark / scikit-learn integration sprint at Cloudera - Strata Conference Friday 14th Feb 2014

Posted by Justin Kestelyn <jk...@cloudera.com>.
Please also add to the venue info: The group will be on the 6th floor.


On Wed, Dec 11, 2013 at 8:14 AM, Horia <ho...@alum.berkeley.edu> wrote:

> If it's up for a vote, I'd say we start at 9:30 :)
>
>
>
> On Wed, Dec 11, 2013 at 1:56 AM, Olivier Grisel <ol...@ensta.org>wrote:
>
>> 2013/12/10 Justin Kestelyn <jk...@cloudera.com>:
>> > Location:
>> >
>> > Cloudera
>> > 433 California Street
>> > San Francisco, CA
>>
>> Thanks I will add that on the wiki, shall we start at 9am? 9.30am?
>>
>> --
>> Olivier
>>
>
>


-- 

*Justin Kestelyn*
Developer Outreach & Relations
Cloudera | www.cloudera.com

Tel: 650.683.4688
Email: jkestelyn@cloudera.com
Twitter: @kestelyn, @ClouderaEng

Re: PySpark / scikit-learn integration sprint at Cloudera - Strata Conference Friday 14th Feb 2014

Posted by Horia <ho...@alum.berkeley.edu>.
If it's up for a vote, I'd say we start at 9:30 :)



On Wed, Dec 11, 2013 at 1:56 AM, Olivier Grisel <ol...@ensta.org>wrote:

> 2013/12/10 Justin Kestelyn <jk...@cloudera.com>:
> > Location:
> >
> > Cloudera
> > 433 California Street
> > San Francisco, CA
>
> Thanks I will add that on the wiki, shall we start at 9am? 9.30am?
>
> --
> Olivier
>

Re: PySpark / scikit-learn integration sprint at Cloudera - Strata Conference Friday 14th Feb 2014

Posted by Olivier Grisel <ol...@ensta.org>.
2013/12/10 Justin Kestelyn <jk...@cloudera.com>:
> Location:
>
> Cloudera
> 433 California Street
> San Francisco, CA

Thanks I will add that on the wiki, shall we start at 9am? 9.30am?

-- 
Olivier

Re: PySpark / scikit-learn integration sprint at Cloudera - Strata Conference Friday 14th Feb 2014

Posted by Justin Kestelyn <jk...@cloudera.com>.
Location:

Cloudera
433 California Street
San Francisco, CA


On Tue, Dec 3, 2013 at 9:30 AM, Horia <ho...@alum.berkeley.edu> wrote:

> I am very interested in this and will most definitely participate!
>
> Please share the event sign-up list and location details when all the
> organizational hurdles have been resolved :-)
>
>
>
> On Mon, Dec 2, 2013 at 1:45 PM, Reynold Xin <rx...@apache.org> wrote:
>
>> Definitely some people will get confused. It's up to you. If we post it,
>> we
>> can mark it in the title that this is a hackathon.
>>
>>
>> On Mon, Dec 2, 2013 at 1:43 PM, Olivier Grisel <olivier.grisel@ensta.org
>> >wrote:
>>
>> > 2013/12/2 Reynold Xin <rx...@apache.org>:
>> > > Including the link to the meetup group:
>> > http://www.meetup.com/spark-users/
>> >
>> > I am not opposed to it but I am wondering if people will not confuse
>> > it with a traditional meetup if we do so.
>> >
>> > --
>> > Olivier
>> >
>>
>
>


-- 

*Justin Kestelyn*
Developer Outreach
Cloudera | www.cloudera.com

Tel: 650.683.4688
Email: jkestelyn@cloudera.com
Twitter: @kestelyn, @ClouderaEng

Re: PySpark / scikit-learn integration sprint at Cloudera - Strata Conference Friday 14th Feb 2014

Posted by Horia <ho...@alum.berkeley.edu>.
I am very interested in this and will most definitely participate!

Please share the event sign-up list and location details when all the
organizational hurdles have been resolved :-)



On Mon, Dec 2, 2013 at 1:45 PM, Reynold Xin <rx...@apache.org> wrote:

> Definitely some people will get confused. It's up to you. If we post it, we
> can mark it in the title that this is a hackathon.
>
>
> On Mon, Dec 2, 2013 at 1:43 PM, Olivier Grisel <olivier.grisel@ensta.org
> >wrote:
>
> > 2013/12/2 Reynold Xin <rx...@apache.org>:
> > > Including the link to the meetup group:
> > http://www.meetup.com/spark-users/
> >
> > I am not opposed to it but I am wondering if people will not confuse
> > it with a traditional meetup if we do so.
> >
> > --
> > Olivier
> >
>

Re: PySpark / scikit-learn integration sprint at Cloudera - Strata Conference Friday 14th Feb 2014

Posted by Reynold Xin <rx...@apache.org>.
Definitely some people will get confused. It's up to you. If we post it, we
can mark it in the title that this is a hackathon.


On Mon, Dec 2, 2013 at 1:43 PM, Olivier Grisel <ol...@ensta.org>wrote:

> 2013/12/2 Reynold Xin <rx...@apache.org>:
> > Including the link to the meetup group:
> http://www.meetup.com/spark-users/
>
> I am not opposed to it but I am wondering if people will not confuse
> it with a traditional meetup if we do so.
>
> --
> Olivier
>

Re: PySpark / scikit-learn integration sprint at Cloudera - Strata Conference Friday 14th Feb 2014

Posted by Olivier Grisel <ol...@ensta.org>.
2013/12/2 Reynold Xin <rx...@apache.org>:
> Including the link to the meetup group: http://www.meetup.com/spark-users/

I am not opposed to it but I am wondering if people will not confuse
it with a traditional meetup if we do so.

-- 
Olivier

Re: PySpark / scikit-learn integration sprint at Cloudera - Strata Conference Friday 14th Feb 2014

Posted by Reynold Xin <rx...@apache.org>.
Including the link to the meetup group: http://www.meetup.com/spark-users/


On Mon, Dec 2, 2013 at 1:22 PM, Reynold Xin <rx...@apache.org> wrote:

> Olivier,
>
> Do you want us to create a Spark user meetup event for this hackathon?
>
> On Mon, Dec 2, 2013 at 1:12 PM, Olivier Grisel <ol...@ensta.org>wrote:
>
>> Hi all,
>>
>> Just a quick reply to say that I would be glad to meet some of you to
>> hack on some prototype scikit-learn / PySpark integration.
>>
>> Cloudera just confirmed that we have a room for us at their San
>> Fransisco offices on Friday Feb 14 (right after Strata).
>>
>> Hope to see you there or at Strata,
>>
>> --
>> Olivier
>>
>
>

Re: PySpark / scikit-learn integration sprint at Cloudera - Strata Conference Friday 14th Feb 2014

Posted by Reynold Xin <rx...@apache.org>.
Olivier,

Do you want us to create a Spark user meetup event for this hackathon?

On Mon, Dec 2, 2013 at 1:12 PM, Olivier Grisel <ol...@ensta.org>wrote:

> Hi all,
>
> Just a quick reply to say that I would be glad to meet some of you to
> hack on some prototype scikit-learn / PySpark integration.
>
> Cloudera just confirmed that we have a room for us at their San
> Fransisco offices on Friday Feb 14 (right after Strata).
>
> Hope to see you there or at Strata,
>
> --
> Olivier
>

Re: PySpark / scikit-learn integration sprint at Cloudera - Strata Conference Friday 14th Feb 2014

Posted by Olivier Grisel <ol...@ensta.org>.
Hi all,

Just a quick reply to say that I would be glad to meet some of you to
hack on some prototype scikit-learn / PySpark integration.

Cloudera just confirmed that we have a room for us at their San
Fransisco offices on Friday Feb 14 (right after Strata).

Hope to see you there or at Strata,

-- 
Olivier