You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by Bessenyei Balázs Donát <be...@apache.org> on 2021/04/02 21:44:56 UTC

Re: GSoC 2021 participation

Thank you for sharing the idea, Adam. I like it!
Can you please share some details?


Donat

On Wed, Mar 31, 2021 at 2:14 AM Adam Kocoloski <ko...@apache.org> wrote:
>
> I would bias towards ecosystem integrations, e.g. a metrics exporter for Prometheus (I think one is in the works), or a  Debezium connector for the _changes feed (I haven’t seen any movement here). I suppose the Debezium connector would ideally be part of the Debezium project itself, but that feels like one that is reasonably self-contained and would provide a good return on investment. Happy to help expand on that one a bit if you think it’s worthwhile.
>
> Adam
>
> > On Mar 30, 2021, at 4:42 AM, Bessenyei Balázs Donát <be...@apache.org> wrote:
> >
> > Wow, I didn't realize we are that far into the timeline.
> >
> > I actually don't have a well-scoped idea for GSoC this year.
> > Does anyone have good candidates for a project?
> >
> >
> > Donat
> >
> > On Mon, Mar 29, 2021 at 6:23 PM Joan Touzet <wo...@apache.org> wrote:
> >>
> >> https://community.apache.org/gsoc.html might be of help.
> >>
> >> If you are looking to talk to other Apache projects that have done this
> >> before, you could reach out on the dev@community.a.o list.
> >>
> >>
> >> https://lists.apache.org/thread.html/r189a563fe003ad8f0e4c298e18fad4da8d4b2854bd2a5d741ae3ac45%40%3Cdev.community.apache.org%3E
> >>
> >> Note the dependency on JIRA:
> >>
> >>> All ASF projects are invited to submit their ideas to their issue tracker, please be sure to add the labels “gsoc2021” and “mentor” so that we can automatically include them in our list of subjects. If your project does not use JIRA please contact dev@community.apache.org.
> >>
> >> so you'll need to post at dev@community to get included in the master list.
> >>
> >> Student applications start tomorrow for 2 weeks, so you'll need to get a
> >> move on... If I'm around on chat I can try and help a bit.
> >>
> >> -Joan
> >>
> >> On 29/03/2021 01:20, Bessenyei Balázs Donát wrote:
> >>> If there are any projects that don't exceed my CouchDB / erlang / JS
> >>> knowledge, I'd make sure I'm available enough to support someone doing
> >>> a GSoC with us.
> >>> What's the workflow here? Do we have to apply as a project? Do we have
> >>> to propose projects?
> >>> I did look at "Prospective ASF mentors: read this" of [1], but I don't
> >>> see what it looks like for a project. Do we need a vote here?
> >>>
> >>>
> >>> Donat
> >>>
> >>> [1]: https://community.apache.org/gsoc.html
> >>>
> >>> On Sun, Mar 28, 2021 at 11:52 PM Joan Touzet <wo...@apache.org> wrote:
> >>>>
> >>>> The ASF often ends up doing GSoC. I don't think we've ever had the
> >>>> sponsor within the project for it (or for Outreachy, for that matter).
> >>>>
> >>>> The most critical part is being available on a regular basis for proper
> >>>> mentoring. If you don't think you can get that into your schedule, don't
> >>>> volunteer. Assume you will get zero support from any other developer
> >>>> (not true, but best to plan for the worst case situation...)
> >>>>
> >>>> The second most critical part is to come up with a self-contained
> >>>> project that makes sense for CouchDB. The most obvious thing to me would
> >>>> be Fauxton work, esp. as it falls into the "sweet spot" of JS
> >>>> development. I dunno how good of a target main is, given how in flux it
> >>>> is; others might have a better take on that. There's also this PR that
> >>>> never finished up:
> >>>>
> >>>>  https://github.com/apache/couchdb/issues/1254
> >>>>
> >>>> These topics are all probably too big, but maybe one of them could be
> >>>> cut down to something summer-sized:
> >>>>
> >>>>  https://github.com/apache/couchdb/projects/1
> >>>>
> >>>> Thanks for taking on this initiative! I know for a fact I won't have
> >>>> time this summer, or I'd agree to join you.
> >>>>
> >>>> -Joan
> >>>>
> >>>> On 28/03/2021 15:59, Bessenyei Balázs Donát wrote:
> >>>>> Hi All,
> >>>>>
> >>>>> I've just seen that the ASF is accepted as a mentoring organisation
> >>>>> for GSoC 2021.
> >>>>> Is CouchDB interested in participating?
> >>>>> I've never done a GSoC before, but I'd certainly be interested. I'd be
> >>>>> happy to help a student contribute to CouchDB.
> >>>>>
> >>>>> What do you all think?
> >>>>>
> >>>>>
> >>>>> Thank you,
> >>>>> Donat
> >>>>>
>

Re: GSoC 2021 participation

Posted by Bessenyei Balázs Donát <be...@apache.org>.
Sounds great, thank you for the details, Adam!
As I'm a PMC member at Apache Flume, I think me helping out with a
Debezium integration is a really good fit for me as I should be
somewhat familiar with streaming log data.
I've posted at dev@community ([1]). Let's see how it goes.


Donat

[1]: https://lists.apache.org/thread.html/rc3a6bc4ca017a76c2e53b94d3faaa2f088963cbe56b608b2e6e4485c%40%3Cdev.community.apache.org%3E

On Sat, Apr 3, 2021 at 12:32 AM Adam Kocoloski <ad...@kocolosk.net> wrote:
>
> Sure thing.
>
> https://debezium.io is an open-source project largely developed by Red Hat that standardizes change data capture across a variety of data sources. It ships with connectors for Oracle, SQL Server, MySQL, PostgreSQL, and MongoDB, and typically records changes into topics in Apache Kafka.
>
> CouchDB, of course, has a change capture feed as a public API endpoint, so a connector should be a smaller effort than consuming the WAL from a traditional RDMS. Integrating with Debezium would provide an easy way to translate the _changes feed into a Kafka topic which plugs us into a much larger ecosystem of tools and alleviates the need for every consumer of data in CouchDB to build a bespoke “follower” of the _changes feed.
>
> Happy to dig up answers to other questions and to make connections with Kafka / Debezium experts where it makes sense. Cheers,
>
> Adam
>
> > On Apr 2, 2021, at 5:44 PM, Bessenyei Balázs Donát <be...@apache.org> wrote:
> >
> > Thank you for sharing the idea, Adam. I like it!
> > Can you please share some details?
> >
> >
> > Donat
> >
> > On Wed, Mar 31, 2021 at 2:14 AM Adam Kocoloski <ko...@apache.org> wrote:
> >>
> >> I would bias towards ecosystem integrations, e.g. a metrics exporter for Prometheus (I think one is in the works), or a  Debezium connector for the _changes feed (I haven’t seen any movement here). I suppose the Debezium connector would ideally be part of the Debezium project itself, but that feels like one that is reasonably self-contained and would provide a good return on investment. Happy to help expand on that one a bit if you think it’s worthwhile.
> >>
> >> Adam
> >>
> >>> On Mar 30, 2021, at 4:42 AM, Bessenyei Balázs Donát <be...@apache.org> wrote:
> >>>
> >>> Wow, I didn't realize we are that far into the timeline.
> >>>
> >>> I actually don't have a well-scoped idea for GSoC this year.
> >>> Does anyone have good candidates for a project?
> >>>
> >>>
> >>> Donat
> >>>
> >>> On Mon, Mar 29, 2021 at 6:23 PM Joan Touzet <wo...@apache.org> wrote:
> >>>>
> >>>> https://community.apache.org/gsoc.html might be of help.
> >>>>
> >>>> If you are looking to talk to other Apache projects that have done this
> >>>> before, you could reach out on the dev@community.a.o list.
> >>>>
> >>>>
> >>>> https://lists.apache.org/thread.html/r189a563fe003ad8f0e4c298e18fad4da8d4b2854bd2a5d741ae3ac45%40%3Cdev.community.apache.org%3E
> >>>>
> >>>> Note the dependency on JIRA:
> >>>>
> >>>>> All ASF projects are invited to submit their ideas to their issue tracker, please be sure to add the labels “gsoc2021” and “mentor” so that we can automatically include them in our list of subjects. If your project does not use JIRA please contact dev@community.apache.org.
> >>>>
> >>>> so you'll need to post at dev@community to get included in the master list.
> >>>>
> >>>> Student applications start tomorrow for 2 weeks, so you'll need to get a
> >>>> move on... If I'm around on chat I can try and help a bit.
> >>>>
> >>>> -Joan
> >>>>
> >>>> On 29/03/2021 01:20, Bessenyei Balázs Donát wrote:
> >>>>> If there are any projects that don't exceed my CouchDB / erlang / JS
> >>>>> knowledge, I'd make sure I'm available enough to support someone doing
> >>>>> a GSoC with us.
> >>>>> What's the workflow here? Do we have to apply as a project? Do we have
> >>>>> to propose projects?
> >>>>> I did look at "Prospective ASF mentors: read this" of [1], but I don't
> >>>>> see what it looks like for a project. Do we need a vote here?
> >>>>>
> >>>>>
> >>>>> Donat
> >>>>>
> >>>>> [1]: https://community.apache.org/gsoc.html
> >>>>>
> >>>>> On Sun, Mar 28, 2021 at 11:52 PM Joan Touzet <wo...@apache.org> wrote:
> >>>>>>
> >>>>>> The ASF often ends up doing GSoC. I don't think we've ever had the
> >>>>>> sponsor within the project for it (or for Outreachy, for that matter).
> >>>>>>
> >>>>>> The most critical part is being available on a regular basis for proper
> >>>>>> mentoring. If you don't think you can get that into your schedule, don't
> >>>>>> volunteer. Assume you will get zero support from any other developer
> >>>>>> (not true, but best to plan for the worst case situation...)
> >>>>>>
> >>>>>> The second most critical part is to come up with a self-contained
> >>>>>> project that makes sense for CouchDB. The most obvious thing to me would
> >>>>>> be Fauxton work, esp. as it falls into the "sweet spot" of JS
> >>>>>> development. I dunno how good of a target main is, given how in flux it
> >>>>>> is; others might have a better take on that. There's also this PR that
> >>>>>> never finished up:
> >>>>>>
> >>>>>> https://github.com/apache/couchdb/issues/1254
> >>>>>>
> >>>>>> These topics are all probably too big, but maybe one of them could be
> >>>>>> cut down to something summer-sized:
> >>>>>>
> >>>>>> https://github.com/apache/couchdb/projects/1
> >>>>>>
> >>>>>> Thanks for taking on this initiative! I know for a fact I won't have
> >>>>>> time this summer, or I'd agree to join you.
> >>>>>>
> >>>>>> -Joan
> >>>>>>
> >>>>>> On 28/03/2021 15:59, Bessenyei Balázs Donát wrote:
> >>>>>>> Hi All,
> >>>>>>>
> >>>>>>> I've just seen that the ASF is accepted as a mentoring organisation
> >>>>>>> for GSoC 2021.
> >>>>>>> Is CouchDB interested in participating?
> >>>>>>> I've never done a GSoC before, but I'd certainly be interested. I'd be
> >>>>>>> happy to help a student contribute to CouchDB.
> >>>>>>>
> >>>>>>> What do you all think?
> >>>>>>>
> >>>>>>>
> >>>>>>> Thank you,
> >>>>>>> Donat
> >>>>>>>
> >>
>

Re: GSoC 2021 participation

Posted by Adam Kocoloski <ad...@kocolosk.net>.
Sure thing.

https://debezium.io is an open-source project largely developed by Red Hat that standardizes change data capture across a variety of data sources. It ships with connectors for Oracle, SQL Server, MySQL, PostgreSQL, and MongoDB, and typically records changes into topics in Apache Kafka.

CouchDB, of course, has a change capture feed as a public API endpoint, so a connector should be a smaller effort than consuming the WAL from a traditional RDMS. Integrating with Debezium would provide an easy way to translate the _changes feed into a Kafka topic which plugs us into a much larger ecosystem of tools and alleviates the need for every consumer of data in CouchDB to build a bespoke “follower” of the _changes feed.

Happy to dig up answers to other questions and to make connections with Kafka / Debezium experts where it makes sense. Cheers,

Adam

> On Apr 2, 2021, at 5:44 PM, Bessenyei Balázs Donát <be...@apache.org> wrote:
> 
> Thank you for sharing the idea, Adam. I like it!
> Can you please share some details?
> 
> 
> Donat
> 
> On Wed, Mar 31, 2021 at 2:14 AM Adam Kocoloski <ko...@apache.org> wrote:
>> 
>> I would bias towards ecosystem integrations, e.g. a metrics exporter for Prometheus (I think one is in the works), or a  Debezium connector for the _changes feed (I haven’t seen any movement here). I suppose the Debezium connector would ideally be part of the Debezium project itself, but that feels like one that is reasonably self-contained and would provide a good return on investment. Happy to help expand on that one a bit if you think it’s worthwhile.
>> 
>> Adam
>> 
>>> On Mar 30, 2021, at 4:42 AM, Bessenyei Balázs Donát <be...@apache.org> wrote:
>>> 
>>> Wow, I didn't realize we are that far into the timeline.
>>> 
>>> I actually don't have a well-scoped idea for GSoC this year.
>>> Does anyone have good candidates for a project?
>>> 
>>> 
>>> Donat
>>> 
>>> On Mon, Mar 29, 2021 at 6:23 PM Joan Touzet <wo...@apache.org> wrote:
>>>> 
>>>> https://community.apache.org/gsoc.html might be of help.
>>>> 
>>>> If you are looking to talk to other Apache projects that have done this
>>>> before, you could reach out on the dev@community.a.o list.
>>>> 
>>>> 
>>>> https://lists.apache.org/thread.html/r189a563fe003ad8f0e4c298e18fad4da8d4b2854bd2a5d741ae3ac45%40%3Cdev.community.apache.org%3E
>>>> 
>>>> Note the dependency on JIRA:
>>>> 
>>>>> All ASF projects are invited to submit their ideas to their issue tracker, please be sure to add the labels “gsoc2021” and “mentor” so that we can automatically include them in our list of subjects. If your project does not use JIRA please contact dev@community.apache.org.
>>>> 
>>>> so you'll need to post at dev@community to get included in the master list.
>>>> 
>>>> Student applications start tomorrow for 2 weeks, so you'll need to get a
>>>> move on... If I'm around on chat I can try and help a bit.
>>>> 
>>>> -Joan
>>>> 
>>>> On 29/03/2021 01:20, Bessenyei Balázs Donát wrote:
>>>>> If there are any projects that don't exceed my CouchDB / erlang / JS
>>>>> knowledge, I'd make sure I'm available enough to support someone doing
>>>>> a GSoC with us.
>>>>> What's the workflow here? Do we have to apply as a project? Do we have
>>>>> to propose projects?
>>>>> I did look at "Prospective ASF mentors: read this" of [1], but I don't
>>>>> see what it looks like for a project. Do we need a vote here?
>>>>> 
>>>>> 
>>>>> Donat
>>>>> 
>>>>> [1]: https://community.apache.org/gsoc.html
>>>>> 
>>>>> On Sun, Mar 28, 2021 at 11:52 PM Joan Touzet <wo...@apache.org> wrote:
>>>>>> 
>>>>>> The ASF often ends up doing GSoC. I don't think we've ever had the
>>>>>> sponsor within the project for it (or for Outreachy, for that matter).
>>>>>> 
>>>>>> The most critical part is being available on a regular basis for proper
>>>>>> mentoring. If you don't think you can get that into your schedule, don't
>>>>>> volunteer. Assume you will get zero support from any other developer
>>>>>> (not true, but best to plan for the worst case situation...)
>>>>>> 
>>>>>> The second most critical part is to come up with a self-contained
>>>>>> project that makes sense for CouchDB. The most obvious thing to me would
>>>>>> be Fauxton work, esp. as it falls into the "sweet spot" of JS
>>>>>> development. I dunno how good of a target main is, given how in flux it
>>>>>> is; others might have a better take on that. There's also this PR that
>>>>>> never finished up:
>>>>>> 
>>>>>> https://github.com/apache/couchdb/issues/1254
>>>>>> 
>>>>>> These topics are all probably too big, but maybe one of them could be
>>>>>> cut down to something summer-sized:
>>>>>> 
>>>>>> https://github.com/apache/couchdb/projects/1
>>>>>> 
>>>>>> Thanks for taking on this initiative! I know for a fact I won't have
>>>>>> time this summer, or I'd agree to join you.
>>>>>> 
>>>>>> -Joan
>>>>>> 
>>>>>> On 28/03/2021 15:59, Bessenyei Balázs Donát wrote:
>>>>>>> Hi All,
>>>>>>> 
>>>>>>> I've just seen that the ASF is accepted as a mentoring organisation
>>>>>>> for GSoC 2021.
>>>>>>> Is CouchDB interested in participating?
>>>>>>> I've never done a GSoC before, but I'd certainly be interested. I'd be
>>>>>>> happy to help a student contribute to CouchDB.
>>>>>>> 
>>>>>>> What do you all think?
>>>>>>> 
>>>>>>> 
>>>>>>> Thank you,
>>>>>>> Donat
>>>>>>> 
>>