You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Wes McKinney <we...@gmail.com> on 2019/02/01 14:15:30 UTC

Google Summer of Code 2019 for Apache Arrow

hi folks,

We are looking for project ideas and mentors for GSoC 2019. I created a JIRA

https://issues.apache.org/jira/browse/COMDEV-309

about a couple of project ideas for the C++ library.

Since Arrow isn't the _easiest_ project to contribute to, we probably
need to calibrate expectations for what a new contributor can
accomplish in a 3 month GSoC project. A good chunk of time at the
beginning will be spent ramping up. If anyone has project ideas (which
need not be in C++) or wants to be a mentor, the deadline is fast
approaching.

Thanks,
Wes

Re: Google Summer of Code 2019 for Apache Arrow

Posted by Wes McKinney <we...@gmail.com>.
I don't have the bandwidth to do more on this right now. If another
member of the community wants to take a leadership role in our
involvement in GSoC that would be really great.

On Wed, Feb 27, 2019 at 1:06 PM Kevin Ratnasekera
<dj...@gmail.com> wrote:
>
> Hi Wes,
>
> I went through the JIRA on [1] there are couple of comments which I can
> make. I dont think you need create ticket on COMDEV, you can create or
> reuse a ticket which is in your project and label it as gsoc2019. I am not
> familiar with Apache Arrow codebase, so I might not be the best person to
> answer this. I personally feel the description provided lacks the
> information needed students get things started. You may provide more what
> you really mentioned as client adapter. Eg:- set of defined interfaces to
> be implemented that should bridge two systems, exact functions which the
> bridge should have. If you already have similar database client adapters
> written for Arrow in other database systems, you can point such
> implementation to the ticket it self. You could also mention if you have
> any documentation of writing such adapters. Most of the student go through
> these ticket will have zero knowledge on what Arrow does. That will help
> students get things started and scope out what needs to be done.
>
> I also prefer projects which has defined scope - projects which has defined
> set of interfaces / classes to be implemented to get job done. These
> projects tend to get successful more. Also better if community have good
> documentation in creating a development setup for the project, building and
> running test for modules. Also good tutorials and sample materials. These
> things help students try the project and come up with proposal.
>
> One thing I want to mention is it not late to even add new project ideas to
> Jira. There s mentor sign up period for Apache ( Which comes shortly ),
> where you can request Apache Org admins to send you an invite to the
> program. Sign up doesnt mean you need to mentor a project. If no student
> show up you may withdraw later, however you will not be able sign up for
> late comers if you have passed the signup period.
>
> I think you don't need to additionally spend time on mentor students. Let
> the students involve with the community / public mailing lists and avoid
> private communication as much as possible. If student is willing to take
> help, there will always be help from community. If you can maintain your
> current involvement with project, it s more than enough to mentor and you
> wont need any addition effort to do this. I also encourage other PMC
> members from Arrow to co mentor with you.
>
> I have involved with GSoC past 5, 6 years above are based on my experience
> with the program, I would personally like to see more students get chance
> to involve with Apache.
>
> [1] https://issues.apache.org/jira/browse/COMDEV-309
> [2] https://issues.apache.org/jira/projects/ARROW
>
> Regards
> Kevin
>
> On Wed, Feb 27, 2019 at 10:14 PM Wes McKinney <we...@gmail.com> wrote:
>
> > hi Kevin,
> >
> > I did post a JIRA ticket about Arrow project ideas. Perhaps I'm
> > mistaken about where we are in the process so hopefully it is still
> > possible to find a student to work on database driver bridges.
> >
> > > I saw your involvement with community and its more than enough time to
> > mentor a student.
> >
> > I have to chuckle a bit about this one =) I personally don't have the
> > bandwidth to do more than I'm already doing.
> >
> > - Wes
> >
> > On Wed, Feb 27, 2019 at 10:38 AM Kevin Ratnasekera
> > <dj...@gmail.com> wrote:
> > >
> > > Hi Wes,
> > >
> > > Why do you think of not participating this year?    Google just announced
> > > apache as accepted org. And this is the usual time where students start
> > to
> > > pop up. Its not too late to create some new ideas and post new tickets on
> > > Jira. It doesn’t even matter you post something very basic, what most
> > > important is to student get chance to involve in a community and learn
> > > something new.
> > >
> > > I saw your involvement with community and its more than enough time to
> > > mentor a student. I encourage you to stay in the program and help
> > students
> > > to get onboarded.
> > >
> > > Regards
> > > Kevin
> > >
> > > On Wed, Feb 27, 2019 at 9:30 PM Wes McKinney <we...@gmail.com>
> > wrote:
> > >
> > > > It would be interesting. I think the ship has sailed on GSoC for us
> > > > unfortunately. I'll try again to get the community interested in it
> > > > next year; hopefully I'll have a little more bandwidth then to help
> > > > make it happen
> > > >
> > > > On Wed, Feb 27, 2019 at 12:13 AM Micah Kornfield <
> > emkornfield@gmail.com>
> > > > wrote:
> > > > >
> > > > > It might be interesting to build an ingestion bridge from Flight
> > servers
> > > > to
> > > > > Spark or vice versa.
> > > > >
> > > > > On Tue, Feb 19, 2019 at 9:25 AM Krisztián Szűcs <
> > > > szucs.krisztian@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > I'd like to implement the Clickhouse[1] bridge, perhaps I can find
> > some
> > > > > > time in the near future. There is a client library[2] which quiet
> > > > nicely
> > > > > > aligns with Arrow's columnar format.
> > > > > > I'd also consider MySQL, because that's the most popular database.
> > > > > >
> > > > > > [1] clickhouse.yandex
> > > > > > [2] https://github.com/artpaul/clickhouse-cpp
> > > > > >
> > > > > > On Tue, Feb 19, 2019 at 5:16 PM Wes McKinney <we...@gmail.com>
> > > > wrote:
> > > > > >
> > > > > > > I agree with Antoine. The more well-defined and less uncertain
> > the
> > > > > > > project, the higher the probability of success. I had suggested
> > > > > > > implementing a bridge between one or more database protocols
> > (e.g.
> > > > > > > SQLite3 or libpq / PostgreSQL) as example projects that could get
> > > > done
> > > > > > > in 3 months. By the way, if anyone is interested in working on
> > these
> > > > > > > projects independent of GSoC please reach out to me.
> > > > > > >
> > > > > > > - Wes
> > > > > > >
> > > > > > > On Tue, Feb 19, 2019 at 3:24 AM Antoine Pitrou <
> > antoine@python.org>
> > > > > > wrote:
> > > > > > > >
> > > > > > > >
> > > > > > > > Le 19/02/2019 à 03:59, Tanya Schlusser a écrit :
> > > > > > > > > Would developing an open standard for in-memory records
> > qualify
> > > > as
> > > > > > > 'GSoC'
> > > > > > > > > worthy?
> > > > > > > > >
> > > > > > > > > In reference to this placeholder in the Confluence wiki:
> > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > >
> > https://cwiki.apache.org/confluence/display/ARROW/Apache+Arrow+Home#ApacheArrowHome-Developinganopenstandardforin-memoryrecords
> > > > > > > > > which links to ARROW-1790
> > > > > > > > >   https://issues.apache.org/jira/browse/ARROW-1790
> > > > > > > > > and to this thread
> > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > >
> > https://lists.apache.org/thread.html/4818cb3d2ffb4677b24a4279c329fc518a1ac1c9d3017399a4269199@%3Cdev.arrow.apache.org%3E
> > > > > > > > >
> > > > > > > > > Developing a standard, or even just starting a standard
> > working
> > > > group
> > > > > > > would
> > > > > > > > > be quite a contribution, and allow a grad student the
> > > > opportunity to
> > > > > > > > > contact multiple leaders in the field. (I am thinking of
> > > > something
> > > > > > > along
> > > > > > > > > the lines of the Data Mining Group http://dmg.org/, which I
> > > > believe
> > > > > > > is run
> > > > > > > > > by a local professor here in Chicago).
> > > > > > > >
> > > > > > > > My indirect experience (I have not mentored a GSoC student,
> > but I
> > > > have
> > > > > > > > followed projects who had GSoC students at some point) is that
> > GSoC
> > > > > > > > projects must be focussed enough, and there should be little
> > to no
> > > > > > > > unknowns, so that the student can progress without getting
> > lost.
> > > > So I
> > > > > > > > don't think asking to develop or start designing a standard is
> > a
> > > > good
> > > > > > > idea.
> > > > > > > >
> > > > > > > > Of course there may be the occasional brillant student who's
> > able
> > > > to
> > > > > > > > overcome all that.
> > > > > > > >
> > > > > > > > Regards
> > > > > > > >
> > > > > > > > Antoine.
> > > > > > >
> > > > > >
> > > >
> >

Re: Google Summer of Code 2019 for Apache Arrow

Posted by Kevin Ratnasekera <dj...@gmail.com>.
Hi Wes,

I went through the JIRA on [1] there are couple of comments which I can
make. I dont think you need create ticket on COMDEV, you can create or
reuse a ticket which is in your project and label it as gsoc2019. I am not
familiar with Apache Arrow codebase, so I might not be the best person to
answer this. I personally feel the description provided lacks the
information needed students get things started. You may provide more what
you really mentioned as client adapter. Eg:- set of defined interfaces to
be implemented that should bridge two systems, exact functions which the
bridge should have. If you already have similar database client adapters
written for Arrow in other database systems, you can point such
implementation to the ticket it self. You could also mention if you have
any documentation of writing such adapters. Most of the student go through
these ticket will have zero knowledge on what Arrow does. That will help
students get things started and scope out what needs to be done.

I also prefer projects which has defined scope - projects which has defined
set of interfaces / classes to be implemented to get job done. These
projects tend to get successful more. Also better if community have good
documentation in creating a development setup for the project, building and
running test for modules. Also good tutorials and sample materials. These
things help students try the project and come up with proposal.

One thing I want to mention is it not late to even add new project ideas to
Jira. There s mentor sign up period for Apache ( Which comes shortly ),
where you can request Apache Org admins to send you an invite to the
program. Sign up doesnt mean you need to mentor a project. If no student
show up you may withdraw later, however you will not be able sign up for
late comers if you have passed the signup period.

I think you don't need to additionally spend time on mentor students. Let
the students involve with the community / public mailing lists and avoid
private communication as much as possible. If student is willing to take
help, there will always be help from community. If you can maintain your
current involvement with project, it s more than enough to mentor and you
wont need any addition effort to do this. I also encourage other PMC
members from Arrow to co mentor with you.

I have involved with GSoC past 5, 6 years above are based on my experience
with the program, I would personally like to see more students get chance
to involve with Apache.

[1] https://issues.apache.org/jira/browse/COMDEV-309
[2] https://issues.apache.org/jira/projects/ARROW

Regards
Kevin

On Wed, Feb 27, 2019 at 10:14 PM Wes McKinney <we...@gmail.com> wrote:

> hi Kevin,
>
> I did post a JIRA ticket about Arrow project ideas. Perhaps I'm
> mistaken about where we are in the process so hopefully it is still
> possible to find a student to work on database driver bridges.
>
> > I saw your involvement with community and its more than enough time to
> mentor a student.
>
> I have to chuckle a bit about this one =) I personally don't have the
> bandwidth to do more than I'm already doing.
>
> - Wes
>
> On Wed, Feb 27, 2019 at 10:38 AM Kevin Ratnasekera
> <dj...@gmail.com> wrote:
> >
> > Hi Wes,
> >
> > Why do you think of not participating this year?    Google just announced
> > apache as accepted org. And this is the usual time where students start
> to
> > pop up. Its not too late to create some new ideas and post new tickets on
> > Jira. It doesn’t even matter you post something very basic, what most
> > important is to student get chance to involve in a community and learn
> > something new.
> >
> > I saw your involvement with community and its more than enough time to
> > mentor a student. I encourage you to stay in the program and help
> students
> > to get onboarded.
> >
> > Regards
> > Kevin
> >
> > On Wed, Feb 27, 2019 at 9:30 PM Wes McKinney <we...@gmail.com>
> wrote:
> >
> > > It would be interesting. I think the ship has sailed on GSoC for us
> > > unfortunately. I'll try again to get the community interested in it
> > > next year; hopefully I'll have a little more bandwidth then to help
> > > make it happen
> > >
> > > On Wed, Feb 27, 2019 at 12:13 AM Micah Kornfield <
> emkornfield@gmail.com>
> > > wrote:
> > > >
> > > > It might be interesting to build an ingestion bridge from Flight
> servers
> > > to
> > > > Spark or vice versa.
> > > >
> > > > On Tue, Feb 19, 2019 at 9:25 AM Krisztián Szűcs <
> > > szucs.krisztian@gmail.com>
> > > > wrote:
> > > >
> > > > > I'd like to implement the Clickhouse[1] bridge, perhaps I can find
> some
> > > > > time in the near future. There is a client library[2] which quiet
> > > nicely
> > > > > aligns with Arrow's columnar format.
> > > > > I'd also consider MySQL, because that's the most popular database.
> > > > >
> > > > > [1] clickhouse.yandex
> > > > > [2] https://github.com/artpaul/clickhouse-cpp
> > > > >
> > > > > On Tue, Feb 19, 2019 at 5:16 PM Wes McKinney <we...@gmail.com>
> > > wrote:
> > > > >
> > > > > > I agree with Antoine. The more well-defined and less uncertain
> the
> > > > > > project, the higher the probability of success. I had suggested
> > > > > > implementing a bridge between one or more database protocols
> (e.g.
> > > > > > SQLite3 or libpq / PostgreSQL) as example projects that could get
> > > done
> > > > > > in 3 months. By the way, if anyone is interested in working on
> these
> > > > > > projects independent of GSoC please reach out to me.
> > > > > >
> > > > > > - Wes
> > > > > >
> > > > > > On Tue, Feb 19, 2019 at 3:24 AM Antoine Pitrou <
> antoine@python.org>
> > > > > wrote:
> > > > > > >
> > > > > > >
> > > > > > > Le 19/02/2019 à 03:59, Tanya Schlusser a écrit :
> > > > > > > > Would developing an open standard for in-memory records
> qualify
> > > as
> > > > > > 'GSoC'
> > > > > > > > worthy?
> > > > > > > >
> > > > > > > > In reference to this placeholder in the Confluence wiki:
> > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > >
> https://cwiki.apache.org/confluence/display/ARROW/Apache+Arrow+Home#ApacheArrowHome-Developinganopenstandardforin-memoryrecords
> > > > > > > > which links to ARROW-1790
> > > > > > > >   https://issues.apache.org/jira/browse/ARROW-1790
> > > > > > > > and to this thread
> > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > >
> https://lists.apache.org/thread.html/4818cb3d2ffb4677b24a4279c329fc518a1ac1c9d3017399a4269199@%3Cdev.arrow.apache.org%3E
> > > > > > > >
> > > > > > > > Developing a standard, or even just starting a standard
> working
> > > group
> > > > > > would
> > > > > > > > be quite a contribution, and allow a grad student the
> > > opportunity to
> > > > > > > > contact multiple leaders in the field. (I am thinking of
> > > something
> > > > > > along
> > > > > > > > the lines of the Data Mining Group http://dmg.org/, which I
> > > believe
> > > > > > is run
> > > > > > > > by a local professor here in Chicago).
> > > > > > >
> > > > > > > My indirect experience (I have not mentored a GSoC student,
> but I
> > > have
> > > > > > > followed projects who had GSoC students at some point) is that
> GSoC
> > > > > > > projects must be focussed enough, and there should be little
> to no
> > > > > > > unknowns, so that the student can progress without getting
> lost.
> > > So I
> > > > > > > don't think asking to develop or start designing a standard is
> a
> > > good
> > > > > > idea.
> > > > > > >
> > > > > > > Of course there may be the occasional brillant student who's
> able
> > > to
> > > > > > > overcome all that.
> > > > > > >
> > > > > > > Regards
> > > > > > >
> > > > > > > Antoine.
> > > > > >
> > > > >
> > >
>

Re: Google Summer of Code 2019 for Apache Arrow

Posted by Wes McKinney <we...@gmail.com>.
hi Kevin,

I did post a JIRA ticket about Arrow project ideas. Perhaps I'm
mistaken about where we are in the process so hopefully it is still
possible to find a student to work on database driver bridges.

> I saw your involvement with community and its more than enough time to mentor a student.

I have to chuckle a bit about this one =) I personally don't have the
bandwidth to do more than I'm already doing.

- Wes

On Wed, Feb 27, 2019 at 10:38 AM Kevin Ratnasekera
<dj...@gmail.com> wrote:
>
> Hi Wes,
>
> Why do you think of not participating this year?    Google just announced
> apache as accepted org. And this is the usual time where students start to
> pop up. Its not too late to create some new ideas and post new tickets on
> Jira. It doesn’t even matter you post something very basic, what most
> important is to student get chance to involve in a community and learn
> something new.
>
> I saw your involvement with community and its more than enough time to
> mentor a student. I encourage you to stay in the program and help students
> to get onboarded.
>
> Regards
> Kevin
>
> On Wed, Feb 27, 2019 at 9:30 PM Wes McKinney <we...@gmail.com> wrote:
>
> > It would be interesting. I think the ship has sailed on GSoC for us
> > unfortunately. I'll try again to get the community interested in it
> > next year; hopefully I'll have a little more bandwidth then to help
> > make it happen
> >
> > On Wed, Feb 27, 2019 at 12:13 AM Micah Kornfield <em...@gmail.com>
> > wrote:
> > >
> > > It might be interesting to build an ingestion bridge from Flight servers
> > to
> > > Spark or vice versa.
> > >
> > > On Tue, Feb 19, 2019 at 9:25 AM Krisztián Szűcs <
> > szucs.krisztian@gmail.com>
> > > wrote:
> > >
> > > > I'd like to implement the Clickhouse[1] bridge, perhaps I can find some
> > > > time in the near future. There is a client library[2] which quiet
> > nicely
> > > > aligns with Arrow's columnar format.
> > > > I'd also consider MySQL, because that's the most popular database.
> > > >
> > > > [1] clickhouse.yandex
> > > > [2] https://github.com/artpaul/clickhouse-cpp
> > > >
> > > > On Tue, Feb 19, 2019 at 5:16 PM Wes McKinney <we...@gmail.com>
> > wrote:
> > > >
> > > > > I agree with Antoine. The more well-defined and less uncertain the
> > > > > project, the higher the probability of success. I had suggested
> > > > > implementing a bridge between one or more database protocols (e.g.
> > > > > SQLite3 or libpq / PostgreSQL) as example projects that could get
> > done
> > > > > in 3 months. By the way, if anyone is interested in working on these
> > > > > projects independent of GSoC please reach out to me.
> > > > >
> > > > > - Wes
> > > > >
> > > > > On Tue, Feb 19, 2019 at 3:24 AM Antoine Pitrou <an...@python.org>
> > > > wrote:
> > > > > >
> > > > > >
> > > > > > Le 19/02/2019 à 03:59, Tanya Schlusser a écrit :
> > > > > > > Would developing an open standard for in-memory records qualify
> > as
> > > > > 'GSoC'
> > > > > > > worthy?
> > > > > > >
> > > > > > > In reference to this placeholder in the Confluence wiki:
> > > > > > >
> > > > > > >
> > > > >
> > > >
> > https://cwiki.apache.org/confluence/display/ARROW/Apache+Arrow+Home#ApacheArrowHome-Developinganopenstandardforin-memoryrecords
> > > > > > > which links to ARROW-1790
> > > > > > >   https://issues.apache.org/jira/browse/ARROW-1790
> > > > > > > and to this thread
> > > > > > >
> > > > > > >
> > > > >
> > > >
> > https://lists.apache.org/thread.html/4818cb3d2ffb4677b24a4279c329fc518a1ac1c9d3017399a4269199@%3Cdev.arrow.apache.org%3E
> > > > > > >
> > > > > > > Developing a standard, or even just starting a standard working
> > group
> > > > > would
> > > > > > > be quite a contribution, and allow a grad student the
> > opportunity to
> > > > > > > contact multiple leaders in the field. (I am thinking of
> > something
> > > > > along
> > > > > > > the lines of the Data Mining Group http://dmg.org/, which I
> > believe
> > > > > is run
> > > > > > > by a local professor here in Chicago).
> > > > > >
> > > > > > My indirect experience (I have not mentored a GSoC student, but I
> > have
> > > > > > followed projects who had GSoC students at some point) is that GSoC
> > > > > > projects must be focussed enough, and there should be little to no
> > > > > > unknowns, so that the student can progress without getting lost.
> > So I
> > > > > > don't think asking to develop or start designing a standard is a
> > good
> > > > > idea.
> > > > > >
> > > > > > Of course there may be the occasional brillant student who's able
> > to
> > > > > > overcome all that.
> > > > > >
> > > > > > Regards
> > > > > >
> > > > > > Antoine.
> > > > >
> > > >
> >

Re: Google Summer of Code 2019 for Apache Arrow

Posted by Kevin Ratnasekera <dj...@gmail.com>.
Hi Wes,

Why do you think of not participating this year?    Google just announced
apache as accepted org. And this is the usual time where students start to
pop up. Its not too late to create some new ideas and post new tickets on
Jira. It doesn’t even matter you post something very basic, what most
important is to student get chance to involve in a community and learn
something new.

I saw your involvement with community and its more than enough time to
mentor a student. I encourage you to stay in the program and help students
to get onboarded.

Regards
Kevin

On Wed, Feb 27, 2019 at 9:30 PM Wes McKinney <we...@gmail.com> wrote:

> It would be interesting. I think the ship has sailed on GSoC for us
> unfortunately. I'll try again to get the community interested in it
> next year; hopefully I'll have a little more bandwidth then to help
> make it happen
>
> On Wed, Feb 27, 2019 at 12:13 AM Micah Kornfield <em...@gmail.com>
> wrote:
> >
> > It might be interesting to build an ingestion bridge from Flight servers
> to
> > Spark or vice versa.
> >
> > On Tue, Feb 19, 2019 at 9:25 AM Krisztián Szűcs <
> szucs.krisztian@gmail.com>
> > wrote:
> >
> > > I'd like to implement the Clickhouse[1] bridge, perhaps I can find some
> > > time in the near future. There is a client library[2] which quiet
> nicely
> > > aligns with Arrow's columnar format.
> > > I'd also consider MySQL, because that's the most popular database.
> > >
> > > [1] clickhouse.yandex
> > > [2] https://github.com/artpaul/clickhouse-cpp
> > >
> > > On Tue, Feb 19, 2019 at 5:16 PM Wes McKinney <we...@gmail.com>
> wrote:
> > >
> > > > I agree with Antoine. The more well-defined and less uncertain the
> > > > project, the higher the probability of success. I had suggested
> > > > implementing a bridge between one or more database protocols (e.g.
> > > > SQLite3 or libpq / PostgreSQL) as example projects that could get
> done
> > > > in 3 months. By the way, if anyone is interested in working on these
> > > > projects independent of GSoC please reach out to me.
> > > >
> > > > - Wes
> > > >
> > > > On Tue, Feb 19, 2019 at 3:24 AM Antoine Pitrou <an...@python.org>
> > > wrote:
> > > > >
> > > > >
> > > > > Le 19/02/2019 à 03:59, Tanya Schlusser a écrit :
> > > > > > Would developing an open standard for in-memory records qualify
> as
> > > > 'GSoC'
> > > > > > worthy?
> > > > > >
> > > > > > In reference to this placeholder in the Confluence wiki:
> > > > > >
> > > > > >
> > > >
> > >
> https://cwiki.apache.org/confluence/display/ARROW/Apache+Arrow+Home#ApacheArrowHome-Developinganopenstandardforin-memoryrecords
> > > > > > which links to ARROW-1790
> > > > > >   https://issues.apache.org/jira/browse/ARROW-1790
> > > > > > and to this thread
> > > > > >
> > > > > >
> > > >
> > >
> https://lists.apache.org/thread.html/4818cb3d2ffb4677b24a4279c329fc518a1ac1c9d3017399a4269199@%3Cdev.arrow.apache.org%3E
> > > > > >
> > > > > > Developing a standard, or even just starting a standard working
> group
> > > > would
> > > > > > be quite a contribution, and allow a grad student the
> opportunity to
> > > > > > contact multiple leaders in the field. (I am thinking of
> something
> > > > along
> > > > > > the lines of the Data Mining Group http://dmg.org/, which I
> believe
> > > > is run
> > > > > > by a local professor here in Chicago).
> > > > >
> > > > > My indirect experience (I have not mentored a GSoC student, but I
> have
> > > > > followed projects who had GSoC students at some point) is that GSoC
> > > > > projects must be focussed enough, and there should be little to no
> > > > > unknowns, so that the student can progress without getting lost.
> So I
> > > > > don't think asking to develop or start designing a standard is a
> good
> > > > idea.
> > > > >
> > > > > Of course there may be the occasional brillant student who's able
> to
> > > > > overcome all that.
> > > > >
> > > > > Regards
> > > > >
> > > > > Antoine.
> > > >
> > >
>

Re: Google Summer of Code 2019 for Apache Arrow

Posted by Wes McKinney <we...@gmail.com>.
It would be interesting. I think the ship has sailed on GSoC for us
unfortunately. I'll try again to get the community interested in it
next year; hopefully I'll have a little more bandwidth then to help
make it happen

On Wed, Feb 27, 2019 at 12:13 AM Micah Kornfield <em...@gmail.com> wrote:
>
> It might be interesting to build an ingestion bridge from Flight servers to
> Spark or vice versa.
>
> On Tue, Feb 19, 2019 at 9:25 AM Krisztián Szűcs <sz...@gmail.com>
> wrote:
>
> > I'd like to implement the Clickhouse[1] bridge, perhaps I can find some
> > time in the near future. There is a client library[2] which quiet nicely
> > aligns with Arrow's columnar format.
> > I'd also consider MySQL, because that's the most popular database.
> >
> > [1] clickhouse.yandex
> > [2] https://github.com/artpaul/clickhouse-cpp
> >
> > On Tue, Feb 19, 2019 at 5:16 PM Wes McKinney <we...@gmail.com> wrote:
> >
> > > I agree with Antoine. The more well-defined and less uncertain the
> > > project, the higher the probability of success. I had suggested
> > > implementing a bridge between one or more database protocols (e.g.
> > > SQLite3 or libpq / PostgreSQL) as example projects that could get done
> > > in 3 months. By the way, if anyone is interested in working on these
> > > projects independent of GSoC please reach out to me.
> > >
> > > - Wes
> > >
> > > On Tue, Feb 19, 2019 at 3:24 AM Antoine Pitrou <an...@python.org>
> > wrote:
> > > >
> > > >
> > > > Le 19/02/2019 à 03:59, Tanya Schlusser a écrit :
> > > > > Would developing an open standard for in-memory records qualify as
> > > 'GSoC'
> > > > > worthy?
> > > > >
> > > > > In reference to this placeholder in the Confluence wiki:
> > > > >
> > > > >
> > >
> > https://cwiki.apache.org/confluence/display/ARROW/Apache+Arrow+Home#ApacheArrowHome-Developinganopenstandardforin-memoryrecords
> > > > > which links to ARROW-1790
> > > > >   https://issues.apache.org/jira/browse/ARROW-1790
> > > > > and to this thread
> > > > >
> > > > >
> > >
> > https://lists.apache.org/thread.html/4818cb3d2ffb4677b24a4279c329fc518a1ac1c9d3017399a4269199@%3Cdev.arrow.apache.org%3E
> > > > >
> > > > > Developing a standard, or even just starting a standard working group
> > > would
> > > > > be quite a contribution, and allow a grad student the opportunity to
> > > > > contact multiple leaders in the field. (I am thinking of something
> > > along
> > > > > the lines of the Data Mining Group http://dmg.org/, which I believe
> > > is run
> > > > > by a local professor here in Chicago).
> > > >
> > > > My indirect experience (I have not mentored a GSoC student, but I have
> > > > followed projects who had GSoC students at some point) is that GSoC
> > > > projects must be focussed enough, and there should be little to no
> > > > unknowns, so that the student can progress without getting lost.  So I
> > > > don't think asking to develop or start designing a standard is a good
> > > idea.
> > > >
> > > > Of course there may be the occasional brillant student who's able to
> > > > overcome all that.
> > > >
> > > > Regards
> > > >
> > > > Antoine.
> > >
> >

Re: Google Summer of Code 2019 for Apache Arrow

Posted by Micah Kornfield <em...@gmail.com>.
It might be interesting to build an ingestion bridge from Flight servers to
Spark or vice versa.

On Tue, Feb 19, 2019 at 9:25 AM Krisztián Szűcs <sz...@gmail.com>
wrote:

> I'd like to implement the Clickhouse[1] bridge, perhaps I can find some
> time in the near future. There is a client library[2] which quiet nicely
> aligns with Arrow's columnar format.
> I'd also consider MySQL, because that's the most popular database.
>
> [1] clickhouse.yandex
> [2] https://github.com/artpaul/clickhouse-cpp
>
> On Tue, Feb 19, 2019 at 5:16 PM Wes McKinney <we...@gmail.com> wrote:
>
> > I agree with Antoine. The more well-defined and less uncertain the
> > project, the higher the probability of success. I had suggested
> > implementing a bridge between one or more database protocols (e.g.
> > SQLite3 or libpq / PostgreSQL) as example projects that could get done
> > in 3 months. By the way, if anyone is interested in working on these
> > projects independent of GSoC please reach out to me.
> >
> > - Wes
> >
> > On Tue, Feb 19, 2019 at 3:24 AM Antoine Pitrou <an...@python.org>
> wrote:
> > >
> > >
> > > Le 19/02/2019 à 03:59, Tanya Schlusser a écrit :
> > > > Would developing an open standard for in-memory records qualify as
> > 'GSoC'
> > > > worthy?
> > > >
> > > > In reference to this placeholder in the Confluence wiki:
> > > >
> > > >
> >
> https://cwiki.apache.org/confluence/display/ARROW/Apache+Arrow+Home#ApacheArrowHome-Developinganopenstandardforin-memoryrecords
> > > > which links to ARROW-1790
> > > >   https://issues.apache.org/jira/browse/ARROW-1790
> > > > and to this thread
> > > >
> > > >
> >
> https://lists.apache.org/thread.html/4818cb3d2ffb4677b24a4279c329fc518a1ac1c9d3017399a4269199@%3Cdev.arrow.apache.org%3E
> > > >
> > > > Developing a standard, or even just starting a standard working group
> > would
> > > > be quite a contribution, and allow a grad student the opportunity to
> > > > contact multiple leaders in the field. (I am thinking of something
> > along
> > > > the lines of the Data Mining Group http://dmg.org/, which I believe
> > is run
> > > > by a local professor here in Chicago).
> > >
> > > My indirect experience (I have not mentored a GSoC student, but I have
> > > followed projects who had GSoC students at some point) is that GSoC
> > > projects must be focussed enough, and there should be little to no
> > > unknowns, so that the student can progress without getting lost.  So I
> > > don't think asking to develop or start designing a standard is a good
> > idea.
> > >
> > > Of course there may be the occasional brillant student who's able to
> > > overcome all that.
> > >
> > > Regards
> > >
> > > Antoine.
> >
>

Re: Google Summer of Code 2019 for Apache Arrow

Posted by Krisztián Szűcs <sz...@gmail.com>.
I'd like to implement the Clickhouse[1] bridge, perhaps I can find some
time in the near future. There is a client library[2] which quiet nicely
aligns with Arrow's columnar format.
I'd also consider MySQL, because that's the most popular database.

[1] clickhouse.yandex
[2] https://github.com/artpaul/clickhouse-cpp

On Tue, Feb 19, 2019 at 5:16 PM Wes McKinney <we...@gmail.com> wrote:

> I agree with Antoine. The more well-defined and less uncertain the
> project, the higher the probability of success. I had suggested
> implementing a bridge between one or more database protocols (e.g.
> SQLite3 or libpq / PostgreSQL) as example projects that could get done
> in 3 months. By the way, if anyone is interested in working on these
> projects independent of GSoC please reach out to me.
>
> - Wes
>
> On Tue, Feb 19, 2019 at 3:24 AM Antoine Pitrou <an...@python.org> wrote:
> >
> >
> > Le 19/02/2019 à 03:59, Tanya Schlusser a écrit :
> > > Would developing an open standard for in-memory records qualify as
> 'GSoC'
> > > worthy?
> > >
> > > In reference to this placeholder in the Confluence wiki:
> > >
> > >
> https://cwiki.apache.org/confluence/display/ARROW/Apache+Arrow+Home#ApacheArrowHome-Developinganopenstandardforin-memoryrecords
> > > which links to ARROW-1790
> > >   https://issues.apache.org/jira/browse/ARROW-1790
> > > and to this thread
> > >
> > >
> https://lists.apache.org/thread.html/4818cb3d2ffb4677b24a4279c329fc518a1ac1c9d3017399a4269199@%3Cdev.arrow.apache.org%3E
> > >
> > > Developing a standard, or even just starting a standard working group
> would
> > > be quite a contribution, and allow a grad student the opportunity to
> > > contact multiple leaders in the field. (I am thinking of something
> along
> > > the lines of the Data Mining Group http://dmg.org/, which I believe
> is run
> > > by a local professor here in Chicago).
> >
> > My indirect experience (I have not mentored a GSoC student, but I have
> > followed projects who had GSoC students at some point) is that GSoC
> > projects must be focussed enough, and there should be little to no
> > unknowns, so that the student can progress without getting lost.  So I
> > don't think asking to develop or start designing a standard is a good
> idea.
> >
> > Of course there may be the occasional brillant student who's able to
> > overcome all that.
> >
> > Regards
> >
> > Antoine.
>

Re: Google Summer of Code 2019 for Apache Arrow

Posted by Wes McKinney <we...@gmail.com>.
I agree with Antoine. The more well-defined and less uncertain the
project, the higher the probability of success. I had suggested
implementing a bridge between one or more database protocols (e.g.
SQLite3 or libpq / PostgreSQL) as example projects that could get done
in 3 months. By the way, if anyone is interested in working on these
projects independent of GSoC please reach out to me.

- Wes

On Tue, Feb 19, 2019 at 3:24 AM Antoine Pitrou <an...@python.org> wrote:
>
>
> Le 19/02/2019 à 03:59, Tanya Schlusser a écrit :
> > Would developing an open standard for in-memory records qualify as 'GSoC'
> > worthy?
> >
> > In reference to this placeholder in the Confluence wiki:
> >
> > https://cwiki.apache.org/confluence/display/ARROW/Apache+Arrow+Home#ApacheArrowHome-Developinganopenstandardforin-memoryrecords
> > which links to ARROW-1790
> >   https://issues.apache.org/jira/browse/ARROW-1790
> > and to this thread
> >
> > https://lists.apache.org/thread.html/4818cb3d2ffb4677b24a4279c329fc518a1ac1c9d3017399a4269199@%3Cdev.arrow.apache.org%3E
> >
> > Developing a standard, or even just starting a standard working group would
> > be quite a contribution, and allow a grad student the opportunity to
> > contact multiple leaders in the field. (I am thinking of something along
> > the lines of the Data Mining Group http://dmg.org/, which I believe is run
> > by a local professor here in Chicago).
>
> My indirect experience (I have not mentored a GSoC student, but I have
> followed projects who had GSoC students at some point) is that GSoC
> projects must be focussed enough, and there should be little to no
> unknowns, so that the student can progress without getting lost.  So I
> don't think asking to develop or start designing a standard is a good idea.
>
> Of course there may be the occasional brillant student who's able to
> overcome all that.
>
> Regards
>
> Antoine.

Re: Google Summer of Code 2019 for Apache Arrow

Posted by Antoine Pitrou <an...@python.org>.
Le 19/02/2019 à 03:59, Tanya Schlusser a écrit :
> Would developing an open standard for in-memory records qualify as 'GSoC'
> worthy?
> 
> In reference to this placeholder in the Confluence wiki:
> 
> https://cwiki.apache.org/confluence/display/ARROW/Apache+Arrow+Home#ApacheArrowHome-Developinganopenstandardforin-memoryrecords
> which links to ARROW-1790
>   https://issues.apache.org/jira/browse/ARROW-1790
> and to this thread
> 
> https://lists.apache.org/thread.html/4818cb3d2ffb4677b24a4279c329fc518a1ac1c9d3017399a4269199@%3Cdev.arrow.apache.org%3E
> 
> Developing a standard, or even just starting a standard working group would
> be quite a contribution, and allow a grad student the opportunity to
> contact multiple leaders in the field. (I am thinking of something along
> the lines of the Data Mining Group http://dmg.org/, which I believe is run
> by a local professor here in Chicago).

My indirect experience (I have not mentored a GSoC student, but I have
followed projects who had GSoC students at some point) is that GSoC
projects must be focussed enough, and there should be little to no
unknowns, so that the student can progress without getting lost.  So I
don't think asking to develop or start designing a standard is a good idea.

Of course there may be the occasional brillant student who's able to
overcome all that.

Regards

Antoine.

Re: Google Summer of Code 2019 for Apache Arrow

Posted by Tanya Schlusser <ta...@tickel.net>.
Would developing an open standard for in-memory records qualify as 'GSoC'
worthy?

In reference to this placeholder in the Confluence wiki:

https://cwiki.apache.org/confluence/display/ARROW/Apache+Arrow+Home#ApacheArrowHome-Developinganopenstandardforin-memoryrecords
which links to ARROW-1790
  https://issues.apache.org/jira/browse/ARROW-1790
and to this thread

https://lists.apache.org/thread.html/4818cb3d2ffb4677b24a4279c329fc518a1ac1c9d3017399a4269199@%3Cdev.arrow.apache.org%3E

Developing a standard, or even just starting a standard working group would
be quite a contribution, and allow a grad student the opportunity to
contact multiple leaders in the field. (I am thinking of something along
the lines of the Data Mining Group http://dmg.org/, which I believe is run
by a local professor here in Chicago).

I don't know many people but can ping that professor and maybe some others
locally if other people think this seems like a GSoC - worthy project.

Best,
Tanya

On Fri, Feb 1, 2019 at 8:16 AM Wes McKinney <we...@gmail.com> wrote:

> hi folks,
>
> We are looking for project ideas and mentors for GSoC 2019. I created a
> JIRA
>
> https://issues.apache.org/jira/browse/COMDEV-309
>
> about a couple of project ideas for the C++ library.
>
> Since Arrow isn't the _easiest_ project to contribute to, we probably
> need to calibrate expectations for what a new contributor can
> accomplish in a 3 month GSoC project. A good chunk of time at the
> beginning will be spent ramping up. If anyone has project ideas (which
> need not be in C++) or wants to be a mentor, the deadline is fast
> approaching.
>
> Thanks,
> Wes
>