You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by Adam Kocoloski <ko...@apache.org> on 2021/10/28 20:44:39 UTC
[DISCUSS] improving visibility for CouchDB-maintained independent
Erlang apps
Hi,
I was doing some work on erlfdb today and I was reminded that we’ve developed a series of Erlang applications that have no dependency on CouchDB on could be generally useful in other Erlang applications. Some examples include:
b64url - NIF for encoding / decoding strings using a url-safe base64 scheme
erlfdb - Erlang bindings for FoundationDB
ets-lru - a bounded LRU cache using ETS tables as the backing store
khash - NIF wrapper around the Kazlib hashing module
snappy - NIF wrapper for Google’s snappy compression library
There are others as well. Each of these libraries is maintained in its own GitHub repo, prefixed with a CouchDB name, e.g.
https://github.com/apache/couchdb-b64url
Unlike the main CouchDB repo, we do not enable Issues or Discussions on these repositories. In many cases the project description and documentation is almost non-existent, e.g. for b64url above the description is “Mirror of Apache CouchDB”.
I think we could benefit from making these projects more visible. Paul’s jiffy library for JSON processing is a nice counterexample where he's gotten non-trivial contributions from the broader Erlang community by putting a little distance between it and CouchDB. Do others agree? If so, I’ve been thinking about some next steps that would help on that front:
- rebar3 compatibility
- project READMEs that clarify the app is independent of (but maintained by) CouchDB
- activating GH Issues (and maybe even Discussions?)
- releases published in Hex? (bit worried about the interaction with ASF release requirements here)
- a page in the CouchDB docs that talks a bit about the Erlang internals and catalogs these apps alongside third-party dependencies
I suppose there are more dramatic steps one could take as well like renaming the repo, but I think it’d be better to start small.
Cheers, Adam
Re: [DISCUSS] improving visibility for CouchDB-maintained independent Erlang apps
Posted by Jan Lehnardt <ja...@apache.org>.
Hi Adam,
> On 19. Nov 2021, at 17:00, Adam Kocoloski <ko...@apache.org> wrote:
>
> Now that we’re getting closer to producing release candidates for some of these component libraries, I’m wondering about where they ought to be uploaded. Maybe a “components” directory alongside the top-level “source” and “binary” directories that we use for CouchDB artifacts? E.g.
>
> source/
> 1.2.2/
> …
> binary/
> mac/
> win/
> components/
> b64url/
> source/
> 1.1.0/
> erlfdb/
> …
>
> Alternatively we could have top-level directories like “couchdb-b64url” as peers to “couchdb", but that seems to head down the path of treating these things as separate projects instead of separate releases, and that’s an overhead I think we’d want to avoid.
>
> Any strong opinions on this front?
The proposal looks good to me.
Best
Jan
—
>
> Adam
>
>> On Nov 19, 2021, at 9:22 AM, Adam Kocoloski <ko...@apache.org> wrote:
>>
>> Intriguing. Looks like Airflow is using GHCR pretty extensively alongside their GHA-based CI. It does make good sense to me to advertise an image alongside the repo as a Quickstart that contains all the dependencies necessary to build the software. Not the most urgent thing, but I’ll add it to the list :)
>>
>> Adam
>>
>>> On Nov 18, 2021, at 6:21 PM, Joan Touzet <wo...@apache.org> wrote:
>>>
>>> FYI: while GH Releases is discouraged for any apache repo, GHCR is apparently acceptable. So perhaps this is an option for your containers?
>>>
>>> https://github.com/apache/yetus/pkgs/container/yetus
>>> https://github.com/apache/yetus/pkgs/container/yetus-base
>>>
>>> reference:
>>>
>>> https://lists.apache.org/thread/66hhhn2t3mx7mg2j9ls4656ngl7j3n0h
>>>
>>> HTH,
>>> Joan
>>>
>>> On 17/11/2021 08:31, Adam Kocoloski wrote:
>>>>> On Nov 17, 2021, at 12:22 AM, Joan Touzet <jo...@jsent.ca> wrote:
>>>>>
>>>>>>> Do we really think these apps are going to have a lot of churn and need a lot of releases?
>>>>>>
>>>>>> I could see `erlfdb` needing a regular release cadence, but I’m willing to see what can be done to comply with the ASF regulations in a semi-automated way. The other repos we’ve been discussing are somewhat more stable, although part of the motivation for publishing packages is to drive more interest in them and that could lead to more releases.
>>>>>>
>>>>>> Tangentially-related question for you: one of the CI jobs for erlfdb is using a container image built from this Dockerfile:
>>>>>>
>>>>>> https://github.com/apache/couchdb-erlfdb/blob/main/.devcontainer/Dockerfile
>>>>>>
>>>>>> Should I publish that image as a tag in apache/couchdbci-debian, even though it’s basically a completely different build than the other images in there? Creating a whole new e.g. apache/erlfdbci repo for it seems like overkill, but I’m curious to hear your thoughts. Would something like this work?
>>>>>>
>>>>>> apache/couchdbci-debian:erlfdb-erlang-24.1.4-fdb-6.3.18
>>>>>>
>>>>>
>>>>> I guess you can't merge it with the other images directly?
>>>>>
>>>>> No objection to using the tagging approach you outline above, of course.
>>>>> But looking at other image names in the apache Docker Hub org, I think
>>>>> Infra would readily approve another name if you need it.
>>>>>
>>>>> -Joan
>>>> I think maybe I could use those existing images? I vaguely recall when I was first building a .devcontainer configuration for CouchDB I tried to start from the images in couchdbdev and ran into an issue I couldn’t sort out. I think it had something to do with the interaction with the erlang_ls plugin for VS Code. But I could take another pass at that at some point. There are a couple of notable differences in the erlfdb image:
>>>> - FDB server is not installed, instead CI configures FDB to run alongside in a service container
>>>> - Image contains a shallow clone of the FDB source code since that’s where the binding tests are defined
>>>> - No extra CouchDB dependencies, e.g. Node, Elixir, SpiderMonkey … probably leads to faster build times
>>>> I do quite like the idea that the .devcontainer and the CI image used for that part of the erlfdb test suite are identical, it just cuts off a whole branch of debugging questions. I’ll go ahead and use a tag like the one above for now, and if erlfdb or other Erlang apps start proliferating we can see about another Docker repository. Thanks,
>>>> Adam
>>
>
Re: [DISCUSS] improving visibility for CouchDB-maintained independent Erlang apps
Posted by Adam Kocoloski <ko...@apache.org>.
Now that we’re getting closer to producing release candidates for some of these component libraries, I’m wondering about where they ought to be uploaded. Maybe a “components” directory alongside the top-level “source” and “binary” directories that we use for CouchDB artifacts? E.g.
source/
1.2.2/
…
binary/
mac/
win/
components/
b64url/
source/
1.1.0/
erlfdb/
…
Alternatively we could have top-level directories like “couchdb-b64url” as peers to “couchdb", but that seems to head down the path of treating these things as separate projects instead of separate releases, and that’s an overhead I think we’d want to avoid.
Any strong opinions on this front?
Adam
> On Nov 19, 2021, at 9:22 AM, Adam Kocoloski <ko...@apache.org> wrote:
>
> Intriguing. Looks like Airflow is using GHCR pretty extensively alongside their GHA-based CI. It does make good sense to me to advertise an image alongside the repo as a Quickstart that contains all the dependencies necessary to build the software. Not the most urgent thing, but I’ll add it to the list :)
>
> Adam
>
>> On Nov 18, 2021, at 6:21 PM, Joan Touzet <wo...@apache.org> wrote:
>>
>> FYI: while GH Releases is discouraged for any apache repo, GHCR is apparently acceptable. So perhaps this is an option for your containers?
>>
>> https://github.com/apache/yetus/pkgs/container/yetus
>> https://github.com/apache/yetus/pkgs/container/yetus-base
>>
>> reference:
>>
>> https://lists.apache.org/thread/66hhhn2t3mx7mg2j9ls4656ngl7j3n0h
>>
>> HTH,
>> Joan
>>
>> On 17/11/2021 08:31, Adam Kocoloski wrote:
>>>> On Nov 17, 2021, at 12:22 AM, Joan Touzet <jo...@jsent.ca> wrote:
>>>>
>>>>>> Do we really think these apps are going to have a lot of churn and need a lot of releases?
>>>>>
>>>>> I could see `erlfdb` needing a regular release cadence, but I’m willing to see what can be done to comply with the ASF regulations in a semi-automated way. The other repos we’ve been discussing are somewhat more stable, although part of the motivation for publishing packages is to drive more interest in them and that could lead to more releases.
>>>>>
>>>>> Tangentially-related question for you: one of the CI jobs for erlfdb is using a container image built from this Dockerfile:
>>>>>
>>>>> https://github.com/apache/couchdb-erlfdb/blob/main/.devcontainer/Dockerfile
>>>>>
>>>>> Should I publish that image as a tag in apache/couchdbci-debian, even though it’s basically a completely different build than the other images in there? Creating a whole new e.g. apache/erlfdbci repo for it seems like overkill, but I’m curious to hear your thoughts. Would something like this work?
>>>>>
>>>>> apache/couchdbci-debian:erlfdb-erlang-24.1.4-fdb-6.3.18
>>>>>
>>>>
>>>> I guess you can't merge it with the other images directly?
>>>>
>>>> No objection to using the tagging approach you outline above, of course.
>>>> But looking at other image names in the apache Docker Hub org, I think
>>>> Infra would readily approve another name if you need it.
>>>>
>>>> -Joan
>>> I think maybe I could use those existing images? I vaguely recall when I was first building a .devcontainer configuration for CouchDB I tried to start from the images in couchdbdev and ran into an issue I couldn’t sort out. I think it had something to do with the interaction with the erlang_ls plugin for VS Code. But I could take another pass at that at some point. There are a couple of notable differences in the erlfdb image:
>>> - FDB server is not installed, instead CI configures FDB to run alongside in a service container
>>> - Image contains a shallow clone of the FDB source code since that’s where the binding tests are defined
>>> - No extra CouchDB dependencies, e.g. Node, Elixir, SpiderMonkey … probably leads to faster build times
>>> I do quite like the idea that the .devcontainer and the CI image used for that part of the erlfdb test suite are identical, it just cuts off a whole branch of debugging questions. I’ll go ahead and use a tag like the one above for now, and if erlfdb or other Erlang apps start proliferating we can see about another Docker repository. Thanks,
>>> Adam
>
Re: [DISCUSS] improving visibility for CouchDB-maintained independent Erlang apps
Posted by Adam Kocoloski <ko...@apache.org>.
Intriguing. Looks like Airflow is using GHCR pretty extensively alongside their GHA-based CI. It does make good sense to me to advertise an image alongside the repo as a Quickstart that contains all the dependencies necessary to build the software. Not the most urgent thing, but I’ll add it to the list :)
Adam
> On Nov 18, 2021, at 6:21 PM, Joan Touzet <wo...@apache.org> wrote:
>
> FYI: while GH Releases is discouraged for any apache repo, GHCR is apparently acceptable. So perhaps this is an option for your containers?
>
> https://github.com/apache/yetus/pkgs/container/yetus
> https://github.com/apache/yetus/pkgs/container/yetus-base
>
> reference:
>
> https://lists.apache.org/thread/66hhhn2t3mx7mg2j9ls4656ngl7j3n0h
>
> HTH,
> Joan
>
> On 17/11/2021 08:31, Adam Kocoloski wrote:
>>> On Nov 17, 2021, at 12:22 AM, Joan Touzet <jo...@jsent.ca> wrote:
>>>
>>>>> Do we really think these apps are going to have a lot of churn and need a lot of releases?
>>>>
>>>> I could see `erlfdb` needing a regular release cadence, but I’m willing to see what can be done to comply with the ASF regulations in a semi-automated way. The other repos we’ve been discussing are somewhat more stable, although part of the motivation for publishing packages is to drive more interest in them and that could lead to more releases.
>>>>
>>>> Tangentially-related question for you: one of the CI jobs for erlfdb is using a container image built from this Dockerfile:
>>>>
>>>> https://github.com/apache/couchdb-erlfdb/blob/main/.devcontainer/Dockerfile
>>>>
>>>> Should I publish that image as a tag in apache/couchdbci-debian, even though it’s basically a completely different build than the other images in there? Creating a whole new e.g. apache/erlfdbci repo for it seems like overkill, but I’m curious to hear your thoughts. Would something like this work?
>>>>
>>>> apache/couchdbci-debian:erlfdb-erlang-24.1.4-fdb-6.3.18
>>>>
>>>
>>> I guess you can't merge it with the other images directly?
>>>
>>> No objection to using the tagging approach you outline above, of course.
>>> But looking at other image names in the apache Docker Hub org, I think
>>> Infra would readily approve another name if you need it.
>>>
>>> -Joan
>> I think maybe I could use those existing images? I vaguely recall when I was first building a .devcontainer configuration for CouchDB I tried to start from the images in couchdbdev and ran into an issue I couldn’t sort out. I think it had something to do with the interaction with the erlang_ls plugin for VS Code. But I could take another pass at that at some point. There are a couple of notable differences in the erlfdb image:
>> - FDB server is not installed, instead CI configures FDB to run alongside in a service container
>> - Image contains a shallow clone of the FDB source code since that’s where the binding tests are defined
>> - No extra CouchDB dependencies, e.g. Node, Elixir, SpiderMonkey … probably leads to faster build times
>> I do quite like the idea that the .devcontainer and the CI image used for that part of the erlfdb test suite are identical, it just cuts off a whole branch of debugging questions. I’ll go ahead and use a tag like the one above for now, and if erlfdb or other Erlang apps start proliferating we can see about another Docker repository. Thanks,
>> Adam
Re: [DISCUSS] improving visibility for CouchDB-maintained independent Erlang apps
Posted by Joan Touzet <wo...@apache.org>.
FYI: while GH Releases is discouraged for any apache repo, GHCR is
apparently acceptable. So perhaps this is an option for your containers?
https://github.com/apache/yetus/pkgs/container/yetus
https://github.com/apache/yetus/pkgs/container/yetus-base
reference:
https://lists.apache.org/thread/66hhhn2t3mx7mg2j9ls4656ngl7j3n0h
HTH,
Joan
On 17/11/2021 08:31, Adam Kocoloski wrote:
>
>> On Nov 17, 2021, at 12:22 AM, Joan Touzet <jo...@jsent.ca> wrote:
>>
>>>> Do we really think these apps are going to have a lot of churn and need a lot of releases?
>>>
>>> I could see `erlfdb` needing a regular release cadence, but I’m willing to see what can be done to comply with the ASF regulations in a semi-automated way. The other repos we’ve been discussing are somewhat more stable, although part of the motivation for publishing packages is to drive more interest in them and that could lead to more releases.
>>>
>>> Tangentially-related question for you: one of the CI jobs for erlfdb is using a container image built from this Dockerfile:
>>>
>>> https://github.com/apache/couchdb-erlfdb/blob/main/.devcontainer/Dockerfile
>>>
>>> Should I publish that image as a tag in apache/couchdbci-debian, even though it’s basically a completely different build than the other images in there? Creating a whole new e.g. apache/erlfdbci repo for it seems like overkill, but I’m curious to hear your thoughts. Would something like this work?
>>>
>>> apache/couchdbci-debian:erlfdb-erlang-24.1.4-fdb-6.3.18
>>>
>>
>> I guess you can't merge it with the other images directly?
>>
>> No objection to using the tagging approach you outline above, of course.
>> But looking at other image names in the apache Docker Hub org, I think
>> Infra would readily approve another name if you need it.
>>
>> -Joan
>
> I think maybe I could use those existing images? I vaguely recall when I was first building a .devcontainer configuration for CouchDB I tried to start from the images in couchdbdev and ran into an issue I couldn’t sort out. I think it had something to do with the interaction with the erlang_ls plugin for VS Code. But I could take another pass at that at some point. There are a couple of notable differences in the erlfdb image:
>
> - FDB server is not installed, instead CI configures FDB to run alongside in a service container
> - Image contains a shallow clone of the FDB source code since that’s where the binding tests are defined
> - No extra CouchDB dependencies, e.g. Node, Elixir, SpiderMonkey … probably leads to faster build times
>
> I do quite like the idea that the .devcontainer and the CI image used for that part of the erlfdb test suite are identical, it just cuts off a whole branch of debugging questions. I’ll go ahead and use a tag like the one above for now, and if erlfdb or other Erlang apps start proliferating we can see about another Docker repository. Thanks,
>
> Adam
>
Re: [DISCUSS] improving visibility for CouchDB-maintained independent Erlang apps
Posted by Adam Kocoloski <ko...@apache.org>.
> On Nov 17, 2021, at 12:22 AM, Joan Touzet <jo...@jsent.ca> wrote:
>
>>> Do we really think these apps are going to have a lot of churn and need a lot of releases?
>>
>> I could see `erlfdb` needing a regular release cadence, but I’m willing to see what can be done to comply with the ASF regulations in a semi-automated way. The other repos we’ve been discussing are somewhat more stable, although part of the motivation for publishing packages is to drive more interest in them and that could lead to more releases.
>>
>> Tangentially-related question for you: one of the CI jobs for erlfdb is using a container image built from this Dockerfile:
>>
>> https://github.com/apache/couchdb-erlfdb/blob/main/.devcontainer/Dockerfile
>>
>> Should I publish that image as a tag in apache/couchdbci-debian, even though it’s basically a completely different build than the other images in there? Creating a whole new e.g. apache/erlfdbci repo for it seems like overkill, but I’m curious to hear your thoughts. Would something like this work?
>>
>> apache/couchdbci-debian:erlfdb-erlang-24.1.4-fdb-6.3.18
>>
>
> I guess you can't merge it with the other images directly?
>
> No objection to using the tagging approach you outline above, of course.
> But looking at other image names in the apache Docker Hub org, I think
> Infra would readily approve another name if you need it.
>
> -Joan
I think maybe I could use those existing images? I vaguely recall when I was first building a .devcontainer configuration for CouchDB I tried to start from the images in couchdbdev and ran into an issue I couldn’t sort out. I think it had something to do with the interaction with the erlang_ls plugin for VS Code. But I could take another pass at that at some point. There are a couple of notable differences in the erlfdb image:
- FDB server is not installed, instead CI configures FDB to run alongside in a service container
- Image contains a shallow clone of the FDB source code since that’s where the binding tests are defined
- No extra CouchDB dependencies, e.g. Node, Elixir, SpiderMonkey … probably leads to faster build times
I do quite like the idea that the .devcontainer and the CI image used for that part of the erlfdb test suite are identical, it just cuts off a whole branch of debugging questions. I’ll go ahead and use a tag like the one above for now, and if erlfdb or other Erlang apps start proliferating we can see about another Docker repository. Thanks,
Adam
Re: [DISCUSS] improving visibility for CouchDB-maintained independent Erlang apps
Posted by Joan Touzet <jo...@jsent.ca>.
>> Do we really think these apps are going to have a lot of churn and need a lot of releases?
>
> I could see `erlfdb` needing a regular release cadence, but I’m willing to see what can be done to comply with the ASF regulations in a semi-automated way. The other repos we’ve been discussing are somewhat more stable, although part of the motivation for publishing packages is to drive more interest in them and that could lead to more releases.
>
> Tangentially-related question for you: one of the CI jobs for erlfdb is using a container image built from this Dockerfile:
>
> https://github.com/apache/couchdb-erlfdb/blob/main/.devcontainer/Dockerfile
>
> Should I publish that image as a tag in apache/couchdbci-debian, even though it’s basically a completely different build than the other images in there? Creating a whole new e.g. apache/erlfdbci repo for it seems like overkill, but I’m curious to hear your thoughts. Would something like this work?
>
> apache/couchdbci-debian:erlfdb-erlang-24.1.4-fdb-6.3.18
>
I guess you can't merge it with the other images directly?
No objection to using the tagging approach you outline above, of course.
But looking at other image names in the apache Docker Hub org, I think
Infra would readily approve another name if you need it.
-Joan
Re: [DISCUSS] improving visibility for CouchDB-maintained independent Erlang apps
Posted by Adam Kocoloski <ko...@apache.org>.
> On Nov 16, 2021, at 6:05 PM, Joan Touzet <wo...@apache.org> wrote:
>
> On 16/11/2021 17:53, Joan Touzet wrote:
>> On 16/11/2021 16:08, Nick Vatamaniuc wrote:
>>> As for voting for making these proper releases, I am not really
>>> looking forward to the SVN pushes, signing, checksumming, and
>>> collecting votes for these deps.
>> Scriptable, shouldn't be hard if someone has time.
>>> Especially for a NIF with just 2
>>> functions in it. Wonder what can be simplified there. Can voting be
>>> done in a GH release draft comments section as opposed a dev@couchdb.
>> No, all votes must be on the mailing list, sorry.
>
> Just to add to this we're not supposed to use the GH release system for any 'official' release, and in fact Infra is in talks with GH to have that system disabled for all apache/* repos. I would absolutely avoid using the GH release system as a basis for the Apache artefacts, and I'd be worried that your scripting today could be undermined tomorrow.
>
> I'm not saying that's right or wrong, it's just what I've heard. If you have concerns, take them up with Infra directly.
>
> Do we really think these apps are going to have a lot of churn and need a lot of releases?
I could see `erlfdb` needing a regular release cadence, but I’m willing to see what can be done to comply with the ASF regulations in a semi-automated way. The other repos we’ve been discussing are somewhat more stable, although part of the motivation for publishing packages is to drive more interest in them and that could lead to more releases.
Tangentially-related question for you: one of the CI jobs for erlfdb is using a container image built from this Dockerfile:
https://github.com/apache/couchdb-erlfdb/blob/main/.devcontainer/Dockerfile
Should I publish that image as a tag in apache/couchdbci-debian, even though it’s basically a completely different build than the other images in there? Creating a whole new e.g. apache/erlfdbci repo for it seems like overkill, but I’m curious to hear your thoughts. Would something like this work?
apache/couchdbci-debian:erlfdb-erlang-24.1.4-fdb-6.3.18
Cheers, Adam
Re: [DISCUSS] improving visibility for CouchDB-maintained independent Erlang apps
Posted by Joan Touzet <wo...@apache.org>.
On 16/11/2021 17:53, Joan Touzet wrote:
> On 16/11/2021 16:08, Nick Vatamaniuc wrote:
>> As for voting for making these proper releases, I am not really
>> looking forward to the SVN pushes, signing, checksumming, and
>> collecting votes for these deps.
>
> Scriptable, shouldn't be hard if someone has time.
>
>> Especially for a NIF with just 2
>> functions in it. Wonder what can be simplified there. Can voting be
>> done in a GH release draft comments section as opposed a dev@couchdb.
>
> No, all votes must be on the mailing list, sorry.
Just to add to this we're not supposed to use the GH release system for
any 'official' release, and in fact Infra is in talks with GH to have
that system disabled for all apache/* repos. I would absolutely avoid
using the GH release system as a basis for the Apache artefacts, and I'd
be worried that your scripting today could be undermined tomorrow.
I'm not saying that's right or wrong, it's just what I've heard. If you
have concerns, take them up with Infra directly.
Do we really think these apps are going to have a lot of churn and need
a lot of releases?
Re: [DISCUSS] improving visibility for CouchDB-maintained independent Erlang apps
Posted by Joan Touzet <wo...@apache.org>.
On 16/11/2021 16:08, Nick Vatamaniuc wrote:
> As for voting for making these proper releases, I am not really
> looking forward to the SVN pushes, signing, checksumming, and
> collecting votes for these deps.
Scriptable, shouldn't be hard if someone has time.
> Especially for a NIF with just 2
> functions in it. Wonder what can be simplified there. Can voting be
> done in a GH release draft comments section as opposed a dev@couchdb.
No, all votes must be on the mailing list, sorry.
> What about SVN pushes, can we automate pushing the GH release .tgz
> artifact to SVN with a bot or something? Or does calling them "tags"
> instead of releases make a difference?
You can automate them but you have to do the signing and signoff
personally, so the best you can do is a script that you run on your
machine with each release, and type in your GPG key to sign it.
> Thanks,
> -Nick
>
> On Tue, Nov 16, 2021 at 3:02 PM Adam Kocoloski <ko...@apache.org> wrote:
>>
>> Hi,
>>
>> I’ve spent some cycles on this work over the past few weeks and wanted to summarize my progress and propose some next steps. I focused on the following two apps:
>>
>> erlfdb (Erlang API for FoundationDB)
>> b64url (Base64 encoder / decoder with URL-safe scheme)
>>
>> I’ve submitted PRs to implement the following functionality from my initial mail:
>>
>>> - rebar3 compatibility
>>> - project READMEs that clarify the app is independent of (but maintained by) CouchDB
>>> - activating GH Issues
>>
>> I’ve also introduced some additional enhancements:
>>
>> - devcontainer configurations to simplify dev environment
>> - CI using GitHub Actions to verify support for Linux + Windows and Erlang/OTP 22-24
>> - branch protection configurations
>> - automated code coverage reports sent to coveralls.io
>>
>> In the few days since I added GH Issues to erlfdb we’ve had a request for packages on hex.pm, which was the next item on the list :) As we discussed, this community should produce official source releases upstream of any packages that get published. I’m willing to take a shot at running a simplified release process for these other repos, though I might need to “phone a friend” the first time through as I’ve never run a CouchDB release. Sound good?
>>
>> In the meantime, I’ve got a few PRs that could use a +1 if folks have the time:
>>
>> https://github.com/apache/couchdb-erlfdb/pull/35
>> https://github.com/apache/couchdb-erlfdb/pull/42
>> https://github.com/apache/couchdb-erlfdb/pull/43
>> https://github.com/apache/couchdb-b64url/pull/8
>>
>> Cheers, Adam
>>
Re: [DISCUSS] improving visibility for CouchDB-maintained independent Erlang apps
Posted by Nick Vatamaniuc <va...@gmail.com>.
Nicely done, Adam!
+1 for most of the PR except one. Perhaps there is a way to push the
CI image to the couchdbdev account instead of your personal one.
As for voting for making these proper releases, I am not really
looking forward to the SVN pushes, signing, checksumming, and
collecting votes for these deps. Especially for a NIF with just 2
functions in it. Wonder what can be simplified there. Can voting be
done in a GH release draft comments section as opposed a dev@couchdb.
What about SVN pushes, can we automate pushing the GH release .tgz
artifact to SVN with a bot or something? Or does calling them "tags"
instead of releases make a difference?
Thanks,
-Nick
On Tue, Nov 16, 2021 at 3:02 PM Adam Kocoloski <ko...@apache.org> wrote:
>
> Hi,
>
> I’ve spent some cycles on this work over the past few weeks and wanted to summarize my progress and propose some next steps. I focused on the following two apps:
>
> erlfdb (Erlang API for FoundationDB)
> b64url (Base64 encoder / decoder with URL-safe scheme)
>
> I’ve submitted PRs to implement the following functionality from my initial mail:
>
> > - rebar3 compatibility
> > - project READMEs that clarify the app is independent of (but maintained by) CouchDB
> > - activating GH Issues
>
> I’ve also introduced some additional enhancements:
>
> - devcontainer configurations to simplify dev environment
> - CI using GitHub Actions to verify support for Linux + Windows and Erlang/OTP 22-24
> - branch protection configurations
> - automated code coverage reports sent to coveralls.io
>
> In the few days since I added GH Issues to erlfdb we’ve had a request for packages on hex.pm, which was the next item on the list :) As we discussed, this community should produce official source releases upstream of any packages that get published. I’m willing to take a shot at running a simplified release process for these other repos, though I might need to “phone a friend” the first time through as I’ve never run a CouchDB release. Sound good?
>
> In the meantime, I’ve got a few PRs that could use a +1 if folks have the time:
>
> https://github.com/apache/couchdb-erlfdb/pull/35
> https://github.com/apache/couchdb-erlfdb/pull/42
> https://github.com/apache/couchdb-erlfdb/pull/43
> https://github.com/apache/couchdb-b64url/pull/8
>
> Cheers, Adam
>
Re: [DISCUSS] improving visibility for CouchDB-maintained independent Erlang apps
Posted by Adam Kocoloski <ko...@apache.org>.
Hi,
I’ve spent some cycles on this work over the past few weeks and wanted to summarize my progress and propose some next steps. I focused on the following two apps:
erlfdb (Erlang API for FoundationDB)
b64url (Base64 encoder / decoder with URL-safe scheme)
I’ve submitted PRs to implement the following functionality from my initial mail:
> - rebar3 compatibility
> - project READMEs that clarify the app is independent of (but maintained by) CouchDB
> - activating GH Issues
I’ve also introduced some additional enhancements:
- devcontainer configurations to simplify dev environment
- CI using GitHub Actions to verify support for Linux + Windows and Erlang/OTP 22-24
- branch protection configurations
- automated code coverage reports sent to coveralls.io
In the few days since I added GH Issues to erlfdb we’ve had a request for packages on hex.pm, which was the next item on the list :) As we discussed, this community should produce official source releases upstream of any packages that get published. I’m willing to take a shot at running a simplified release process for these other repos, though I might need to “phone a friend” the first time through as I’ve never run a CouchDB release. Sound good?
In the meantime, I’ve got a few PRs that could use a +1 if folks have the time:
https://github.com/apache/couchdb-erlfdb/pull/35
https://github.com/apache/couchdb-erlfdb/pull/42
https://github.com/apache/couchdb-erlfdb/pull/43
https://github.com/apache/couchdb-b64url/pull/8
Cheers, Adam
Re: [DISCUSS] improving visibility for CouchDB-maintained independent
Erlang apps
Posted by Adam Kocoloski <ko...@apache.org>.
> On Oct 29, 2021, at 4:03 PM, Nick Vatamaniuc <va...@gmail.com> wrote:
>
> Technically there are difficulties around rebar3 compatibility for
> NIFs. It might be easier to make them developer friendly after we
> switched the whole CouchDB project to rebar3.
I recall that discussion cropping up in the past. I managed to get rebar3’s port_compiler plugin to support erlfdb in this PR:
https://github.com/apache/couchdb-erlfdb/pull/29 <https://github.com/apache/couchdb-erlfdb/pull/29>
As far as I can tell the `pc` plugin does basically the same thing as `enc`; i.e., hack and slash through rebar2 to extract the parts that are relevant to native code compilation, while preserving the configuration interface. Not sure if there are other issues lurking.
> There is also another direction we can take some applications like
> config, for example. I think it might make sense to pull that one into
> the main repo. Over the years it got tangled enough with CouchDB
> specifics (couch_log, etc) that having it as separate is more of an
> inconvenience than a benefit at this point.
I’d probably defer doing anything with config till we tackle some of the simpler cases, but I am definitely open to folding it into CouchDB and giving up on the idea of it as a general-purpose app. Cheers,
Adam
Re: [DISCUSS] improving visibility for CouchDB-maintained independent
Erlang apps
Posted by Nick Vatamaniuc <va...@gmail.com>.
Good idea, Adam. Some of those are pretty nice applications.
Technically there are difficulties around rebar3 compatibility for
NIFs. It might be easier to make them developer friendly after we
switched the whole CouchDB project to rebar3. The non-NIF apps can be
used already as source dependencies. For example: {ets_lru, {git,
"https://github.com/apache/couchdb-ets-lru.git", {tag, "1.1.0"}}}. We
also create git tags when we update them so it's easy to refer to
those, instead of just plain commit ids. But documentation is still
missing and that would be nice to add.
Regarding releases, anyone can clone these repos and use them, but I
guess we cannot say they are ASF project "releases". Is that a correct
interpretation? Not sure if adding a disclaimer to the docs might
help, too.
There is also another direction we can take some applications like
config, for example. I think it might make sense to pull that one into
the main repo. Over the years it got tangled enough with CouchDB
specifics (couch_log, etc) that having it as separate is more of an
inconvenience than a benefit at this point.
Cheers,
-Nick
On Fri, Oct 29, 2021 at 9:53 AM Adam Kocoloski <ko...@apache.org> wrote:
>
> Ah! That .asf.yaml configuration is neat, thanks for pointing it out.
>
> Good point on the source releases — do you have anything in particular in mind when you talk about making it simpler for the community at large?
>
> Adam
>
> > On Oct 28, 2021, at 10:48 PM, Joan Touzet <wo...@apache.org> wrote:
> >
> > On 28/10/2021 16:44, Adam Kocoloski wrote:
> >> I think we could benefit from making these projects more visible. Paul’s jiffy library for JSON processing is a nice counterexample where he's gotten non-trivial contributions from the broader Erlang community by putting a little distance between it and CouchDB. Do others agree?
> >
> > No opinion here, but:
> >
> >> If so, I’ve been thinking about some next steps that would help on that front:
> >> - activating GH Issues (and maybe even Discussions?)
> >
> > The only ASF thing is that we must ensure that all of the Issues traffic ends up on a mailing list for archival purposes. (I think Infra are still working on mirroring Discussions traffic.)
> >
> > Mirroring that traffic and enabling issues is in fact self-serve now:
> >
> > https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features#Git.asf.yamlfeatures-Notificationsettingsforrepositories
> >
> > and
> >
> > https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features#Git.asf.yamlfeatures-Repositoryfeatures
> >
> > If you want Discussions you have to open an Infra ticket.
> >
> >> - releases published in Hex? (bit worried about the interaction with ASF release requirements here)
> >
> > "Real" source releases would still have to go through voting and 3 +1 PMC votes - this could be made simpler for the community at large. with documentation on how to test these applications independently of CouchDB.
> >
> > Binary releases aren't "real" releases, but they would need to be paired with actual ASF releases due to policy. No problem with them being on Hex just as there's no problem with us being on Docker or any other binary stream, just so long as we do the "official" dance first.
> >
> > -Joan
>
Re: [DISCUSS] improving visibility for CouchDB-maintained independent
Erlang apps
Posted by Joan Touzet <wo...@apache.org>.
On 29/10/2021 09:52, Adam Kocoloski wrote:
> Ah! That .asf.yaml configuration is neat, thanks for pointing it out.
>
> Good point on the source releases — do you have anything in particular in mind when you talk about making it simpler for the community at large?
Sorry, that . should be a , :
> this could be made simpler for the community at large, with documentation on how to test these applications independently of CouchDB.
meaning just make the release process easier to vote on. Not knowing
anything about those apps, I'd want docs on how to put it thru its paces
to feel right about voting on it. If we're happy with just passing unit
tests, then that's that.
Nick also writes:
> Regarding releases, anyone can clone these repos and use them, but I
> guess we cannot say they are ASF project "releases". Is that a correct
> interpretation? Not sure if adding a disclaimer to the docs might
> help, too.
Anyone can clone and use CouchDB today too! Doesn't stop us from doing
proper releases on it. If you want to call them valuable enough to be
standalone things, and release them on Hex as binaries, then we should
do the right thing and treat them as first-class citizens. And with good
docs we can make this as painless a process as possible for everyone,
modulo having to use svn to upload the tarballs after voting.
-Joan
Re: [DISCUSS] improving visibility for CouchDB-maintained independent
Erlang apps
Posted by Adam Kocoloski <ko...@apache.org>.
Ah! That .asf.yaml configuration is neat, thanks for pointing it out.
Good point on the source releases — do you have anything in particular in mind when you talk about making it simpler for the community at large?
Adam
> On Oct 28, 2021, at 10:48 PM, Joan Touzet <wo...@apache.org> wrote:
>
> On 28/10/2021 16:44, Adam Kocoloski wrote:
>> I think we could benefit from making these projects more visible. Paul’s jiffy library for JSON processing is a nice counterexample where he's gotten non-trivial contributions from the broader Erlang community by putting a little distance between it and CouchDB. Do others agree?
>
> No opinion here, but:
>
>> If so, I’ve been thinking about some next steps that would help on that front:
>> - activating GH Issues (and maybe even Discussions?)
>
> The only ASF thing is that we must ensure that all of the Issues traffic ends up on a mailing list for archival purposes. (I think Infra are still working on mirroring Discussions traffic.)
>
> Mirroring that traffic and enabling issues is in fact self-serve now:
>
> https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features#Git.asf.yamlfeatures-Notificationsettingsforrepositories
>
> and
>
> https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features#Git.asf.yamlfeatures-Repositoryfeatures
>
> If you want Discussions you have to open an Infra ticket.
>
>> - releases published in Hex? (bit worried about the interaction with ASF release requirements here)
>
> "Real" source releases would still have to go through voting and 3 +1 PMC votes - this could be made simpler for the community at large. with documentation on how to test these applications independently of CouchDB.
>
> Binary releases aren't "real" releases, but they would need to be paired with actual ASF releases due to policy. No problem with them being on Hex just as there's no problem with us being on Docker or any other binary stream, just so long as we do the "official" dance first.
>
> -Joan
Re: [DISCUSS] improving visibility for CouchDB-maintained independent
Erlang apps
Posted by Joan Touzet <wo...@apache.org>.
On 28/10/2021 16:44, Adam Kocoloski wrote:
> I think we could benefit from making these projects more visible. Paul’s jiffy library for JSON processing is a nice counterexample where he's gotten non-trivial contributions from the broader Erlang community by putting a little distance between it and CouchDB. Do others agree?
No opinion here, but:
> If so, I’ve been thinking about some next steps that would help on that front:
> - activating GH Issues (and maybe even Discussions?)
The only ASF thing is that we must ensure that all of the Issues traffic
ends up on a mailing list for archival purposes. (I think Infra are
still working on mirroring Discussions traffic.)
Mirroring that traffic and enabling issues is in fact self-serve now:
https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features#Git.asf.yamlfeatures-Notificationsettingsforrepositories
and
https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features#Git.asf.yamlfeatures-Repositoryfeatures
If you want Discussions you have to open an Infra ticket.
> - releases published in Hex? (bit worried about the interaction with ASF release requirements here)
"Real" source releases would still have to go through voting and 3 +1
PMC votes - this could be made simpler for the community at large. with
documentation on how to test these applications independently of CouchDB.
Binary releases aren't "real" releases, but they would need to be paired
with actual ASF releases due to policy. No problem with them being on
Hex just as there's no problem with us being on Docker or any other
binary stream, just so long as we do the "official" dance first.
-Joan