You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@iggy.apache.org by tison <wa...@gmail.com> on 2025/02/07 03:30:01 UTC

[DISCUSS] Monorepo vs. Multiple Repositories

Hi,

I'd continue the discussion from [1] about whether we'd set up a
monorepo for all the Iggy's code, or continue the current multiple
repositories solution.

[1] https://lists.apache.org/thread/ppqjvygk8xs7v8no5dk6r6khm5m1vp71

The only thing I'd point out is that the ASF doesn't support
fine-grained permissions so please make correct assumption.

Either monorepo (OpenDAL's multilingual bindings [2]) or multiple
repos (SkyWalking and its multilingual SDK [3]) works for me.

[2] https://github.com/apache/opendal
[3] https://github.com/apache?q=skywalking&type=all&language=&sort=

Best,
tison.

Re: [DISCUSS] Monorepo vs. Multiple Repositories

Posted by Kranti Parisa <kr...@apache.org>.
Great, we can move fwd with single repo then.

@Xuanwo once we have the repo in place, we would highly appreciate your
help in setting up the CI/CD pipelines like you did for OpenDAL.

I've opened the Jira ticket for moving git repo.

Best,
Kranti



On Sat, Feb 8, 2025 at 12:30 PM Hubert Gruszecki <hg...@apache.org>
wrote:

> Also +1 for monorepo from my side. We could nicely integrate tests for all
> SDKs in other languages so they are never out of sync compared to Rust SDK,
> which is was always a first class citizen.
>
> On 2025/02/08 08:00:10 Grzegorz Koszyk wrote:
> > +1 for the monorepo from me, we will in the future for sure move towards
> bindings for our sdks (not sure if entirely or partially), having it in a
> single repository makes the migrating/maintaining process much easier.
> >
> > On 2025/02/07 03:30:01 tison wrote:
> > > Hi,
> > >
> > > I'd continue the discussion from [1] about whether we'd set up a
> > > monorepo for all the Iggy's code, or continue the current multiple
> > > repositories solution.
> > >
> > > [1] https://lists.apache.org/thread/ppqjvygk8xs7v8no5dk6r6khm5m1vp71
> > >
> > > The only thing I'd point out is that the ASF doesn't support
> > > fine-grained permissions so please make correct assumption.
> > >
> > > Either monorepo (OpenDAL's multilingual bindings [2]) or multiple
> > > repos (SkyWalking and its multilingual SDK [3]) works for me.
> > >
> > > [2] https://github.com/apache/opendal
> > > [3] https://github.com/apache?q=skywalking&type=all&language=&sort=
> > >
> > > Best,
> > > tison.
> > >
> >
>

Re: [DISCUSS] Monorepo vs. Multiple Repositories

Posted by Hubert Gruszecki <hg...@apache.org>.
Also +1 for monorepo from my side. We could nicely integrate tests for all SDKs in other languages so they are never out of sync compared to Rust SDK, which is was always a first class citizen.

On 2025/02/08 08:00:10 Grzegorz Koszyk wrote:
> +1 for the monorepo from me, we will in the future for sure move towards bindings for our sdks (not sure if entirely or partially), having it in a single repository makes the migrating/maintaining process much easier.
> 
> On 2025/02/07 03:30:01 tison wrote:
> > Hi,
> > 
> > I'd continue the discussion from [1] about whether we'd set up a
> > monorepo for all the Iggy's code, or continue the current multiple
> > repositories solution.
> > 
> > [1] https://lists.apache.org/thread/ppqjvygk8xs7v8no5dk6r6khm5m1vp71
> > 
> > The only thing I'd point out is that the ASF doesn't support
> > fine-grained permissions so please make correct assumption.
> > 
> > Either monorepo (OpenDAL's multilingual bindings [2]) or multiple
> > repos (SkyWalking and its multilingual SDK [3]) works for me.
> > 
> > [2] https://github.com/apache/opendal
> > [3] https://github.com/apache?q=skywalking&type=all&language=&sort=
> > 
> > Best,
> > tison.
> > 
> 

Re: [DISCUSS] Monorepo vs. Multiple Repositories

Posted by Grzegorz Koszyk <gk...@apache.org>.
+1 for the monorepo from me, we will in the future for sure move towards bindings for our sdks (not sure if entirely or partially), having it in a single repository makes the migrating/maintaining process much easier.

On 2025/02/07 03:30:01 tison wrote:
> Hi,
> 
> I'd continue the discussion from [1] about whether we'd set up a
> monorepo for all the Iggy's code, or continue the current multiple
> repositories solution.
> 
> [1] https://lists.apache.org/thread/ppqjvygk8xs7v8no5dk6r6khm5m1vp71
> 
> The only thing I'd point out is that the ASF doesn't support
> fine-grained permissions so please make correct assumption.
> 
> Either monorepo (OpenDAL's multilingual bindings [2]) or multiple
> repos (SkyWalking and its multilingual SDK [3]) works for me.
> 
> [2] https://github.com/apache/opendal
> [3] https://github.com/apache?q=skywalking&type=all&language=&sort=
> 
> Best,
> tison.
> 

Re: [DISCUSS] Monorepo vs. Multiple Repositories

Posted by Kranti Parisa <kr...@apache.org>.
Agreed, there are pros and cons. But I like Xuanwo's thoughts around growing the project much smoother way.

Our biggest concern was around CI/CD, so if we could spend some time upfront to setup things, it should be on autopilot mode after.

Are there any examples of such mono repos with different CI/CD pipelines to release SDKs and main server independently?

On 2025/02/07 06:50:53 tison wrote:
> The downside of mono repo is that CI/CD, especially releases, would be
> much more complicated. It's possible but you need some time to work on
> tools and settings.
> 
> Best,
> tison.
> 
> tison <wa...@gmail.com> 于2025年2月7日周五 14:49写道:
> >
> > EDIT: [1] https://github.com/apache/infrastructure-asfyaml/blob/main/README.md
> >
> > Best,
> > tison.
> >
> > tison <wa...@gmail.com> 于2025年2月7日周五 14:48写道:
> > >
> > > > First of all, after migrating the repositories to the ASF, we will lose admin permissions for them. This means that any operations requiring admin privileges will have to go through INFRA tickets unless they are supported by ASF tools.
> > >
> > > To clarify, the INFRA team is working on making the experience more
> > > smooth, check [1][2].
> > >
> > > [1] https://github.com/apache/infrastructure-asfyaml/blob/ng-parser/docs/ASFYamlFeature.md
> > > [2] https://github.com/apache/infrastructure-asfyaml/blob/ng-parser/docs/ASFYamlFeature.md
> > >
> > > But .. yes, it's not GitHub-Native.
> > >
> > > Best,
> > > tison.
> > >
> > > Xuanwo <xu...@apache.org> 于2025年2月7日周五 14:45写道:
> > > >
> > > > I'd like to share some reasons why I prefer using a monorepo for OpenDAL.
> > > >
> > > > First of all, after migrating the repositories to the ASF, we will lose admin permissions for them. This means that any operations requiring admin privileges will have to go through INFRA tickets unless they are supported by ASF tools. For example, configuring GitHub secrets or making CI run for every PR by default. Setting this up for multiple repositories could be very challenging.
> > > >
> > > > Secondly, I find that keeping everything in the same monorepo makes it easier for the project to grow. Stars, issues, PRs, and discussions all happen in the same repository, making it easier to bring people together. This is especially important for a small and new project. We can always split the project later once we have a sufficient number of dedicated committers focusing on a specific area—for example, three committers who explicitly dedicate themselves to the Go SDK.
> > > >
> > > > Both setups work for me.
> > > >
> > > > On Fri, Feb 7, 2025, at 14:28, tison wrote:
> > > > > cc in case you don't subscribe dev@ yet.
> > > > >
> > > > > You can reply with PonyMail also [1]
> > > > >
> > > > > [1] https://lists.apache.org/thread/zrn96nlg23r9353lr5tp2by2ggx4zxqc
> > > > >
> > > > > Best,
> > > > > tison.
> > > > >
> > > > > tison <wa...@gmail.com> 于2025年2月7日周五 11:30写道:
> > > > >
> > > > >>
> > > > >> Hi,
> > > > >>
> > > > >> I'd continue the discussion from [1] about whether we'd set up a
> > > > >> monorepo for all the Iggy's code, or continue the current multiple
> > > > >> repositories solution.
> > > > >>
> > > > >> [1] https://lists.apache.org/thread/ppqjvygk8xs7v8no5dk6r6khm5m1vp71
> > > > >>
> > > > >> The only thing I'd point out is that the ASF doesn't support
> > > > >> fine-grained permissions so please make correct assumption.
> > > > >>
> > > > >> Either monorepo (OpenDAL's multilingual bindings [2]) or multiple
> > > > >> repos (SkyWalking and its multilingual SDK [3]) works for me.
> > > > >>
> > > > >> [2] https://github.com/apache/opendal
> > > > >> [3] https://github.com/apache?q=skywalking&type=all&language=&sort=
> > > > >>
> > > > >> Best,
> > > > >> tison.
> > > >
> > > > --
> > > > Xuanwo
> > > >
> > > > https://xuanwo.io/
> 

Re: [DISCUSS] Monorepo vs. Multiple Repositories

Posted by Xuanwo <xu...@apache.org>.
That looks good to me.

On Sat, Feb 8, 2025, at 15:50, Piotr Gankiewicz wrote:
> In case of using the monorepo, could we do this in the following way:
> - Move our main iggy-rs/iggy repo first under apache/iggy
> - Work on structuring the new monorepo under apache/iggy to make sure we
> won't break the already configured pipelines etc.
> - Start moving the remaining repos under iggy-rs such as client SDKs and
> other tooling to the apache/iggy monorepo once we figure out how to
> structure subdirectories and the other pipelines to make it all work
> - Update the readme files for all the repos under iggy-rs that all the
> codebase now lives under a single monorepo at apache/iggy
>
>
> On Sat, 8 Feb 2025 at 04:57, tison <wa...@gmail.com> wrote:
>
>> Note that OpenDAL is currently releasing simulately, while each
>> component can have different version.
>>
>> To release multilingual SDK and the main server, you should determine:
>>
>> 1. Version strategy (tag strategy)
>> 2. How to manually/automatically deploy to the target repository
>> (crates.io, Maven, PyPI, NPM etc.)
>> 3. The release process to fit in ASF requirements [1][2]
>>
>> [1] https://www.apache.org/legal/release-policy.html
>> [2] https://opendal.apache.org/community/release/
>>
>> Best,
>> tison.
>>
>> Xuanwo <xu...@apache.org> 于2025年2月8日周六 11:16写道:
>> >
>> > Welcome to check opendal's CI:
>> https://github.com/apache/opendal/tree/main/.github/workflows
>> >
>> > On Sat, Feb 8, 2025, at 11:15, Kranti Parisa wrote:
>> > > Agreed, there are pros and cons. But I like Xuanwo's thoughts around
>> > > growing the project much smoother way.
>> > >
>> > > Our biggest concern was around CI/CD, so if we could spend some time
>> > > upfront to setup things, it should be on autopilot mode after.
>> > >
>> > > Are there any examples of such mono repos with different CI/CD
>> > > pipelines to release SDKs and main server independently?
>> > >
>> > > On 2025/02/07 06:50:53 tison wrote:
>> > >> The downside of mono repo is that CI/CD, especially releases, would be
>> > >> much more complicated. It's possible but you need some time to work on
>> > >> tools and settings.
>> > >>
>> > >> Best,
>> > >> tison.
>> > >>
>> > >> tison <wa...@gmail.com> 于2025年2月7日周五 14:49写道:
>> > >> >
>> > >> > EDIT: [1]
>> https://github.com/apache/infrastructure-asfyaml/blob/main/README.md
>> > >> >
>> > >> > Best,
>> > >> > tison.
>> > >> >
>> > >> > tison <wa...@gmail.com> 于2025年2月7日周五 14:48写道:
>> > >> > >
>> > >> > > > First of all, after migrating the repositories to the ASF, we
>> will lose admin permissions for them. This means that any operations
>> requiring admin privileges will have to go through INFRA tickets unless
>> they are supported by ASF tools.
>> > >> > >
>> > >> > > To clarify, the INFRA team is working on making the experience
>> more
>> > >> > > smooth, check [1][2].
>> > >> > >
>> > >> > > [1]
>> https://github.com/apache/infrastructure-asfyaml/blob/ng-parser/docs/ASFYamlFeature.md
>> > >> > > [2]
>> https://github.com/apache/infrastructure-asfyaml/blob/ng-parser/docs/ASFYamlFeature.md
>> > >> > >
>> > >> > > But .. yes, it's not GitHub-Native.
>> > >> > >
>> > >> > > Best,
>> > >> > > tison.
>> > >> > >
>> > >> > > Xuanwo <xu...@apache.org> 于2025年2月7日周五 14:45写道:
>> > >> > > >
>> > >> > > > I'd like to share some reasons why I prefer using a monorepo
>> for OpenDAL.
>> > >> > > >
>> > >> > > > First of all, after migrating the repositories to the ASF, we
>> will lose admin permissions for them. This means that any operations
>> requiring admin privileges will have to go through INFRA tickets unless
>> they are supported by ASF tools. For example, configuring GitHub secrets or
>> making CI run for every PR by default. Setting this up for multiple
>> repositories could be very challenging.
>> > >> > > >
>> > >> > > > Secondly, I find that keeping everything in the same monorepo
>> makes it easier for the project to grow. Stars, issues, PRs, and
>> discussions all happen in the same repository, making it easier to bring
>> people together. This is especially important for a small and new project.
>> We can always split the project later once we have a sufficient number of
>> dedicated committers focusing on a specific area—for example, three
>> committers who explicitly dedicate themselves to the Go SDK.
>> > >> > > >
>> > >> > > > Both setups work for me.
>> > >> > > >
>> > >> > > > On Fri, Feb 7, 2025, at 14:28, tison wrote:
>> > >> > > > > cc in case you don't subscribe dev@ yet.
>> > >> > > > >
>> > >> > > > > You can reply with PonyMail also [1]
>> > >> > > > >
>> > >> > > > > [1]
>> https://lists.apache.org/thread/zrn96nlg23r9353lr5tp2by2ggx4zxqc
>> > >> > > > >
>> > >> > > > > Best,
>> > >> > > > > tison.
>> > >> > > > >
>> > >> > > > > tison <wa...@gmail.com> 于2025年2月7日周五 11:30写道:
>> > >> > > > >
>> > >> > > > >>
>> > >> > > > >> Hi,
>> > >> > > > >>
>> > >> > > > >> I'd continue the discussion from [1] about whether we'd set
>> up a
>> > >> > > > >> monorepo for all the Iggy's code, or continue the current
>> multiple
>> > >> > > > >> repositories solution.
>> > >> > > > >>
>> > >> > > > >> [1]
>> https://lists.apache.org/thread/ppqjvygk8xs7v8no5dk6r6khm5m1vp71
>> > >> > > > >>
>> > >> > > > >> The only thing I'd point out is that the ASF doesn't support
>> > >> > > > >> fine-grained permissions so please make correct assumption.
>> > >> > > > >>
>> > >> > > > >> Either monorepo (OpenDAL's multilingual bindings [2]) or
>> multiple
>> > >> > > > >> repos (SkyWalking and its multilingual SDK [3]) works for me.
>> > >> > > > >>
>> > >> > > > >> [2] https://github.com/apache/opendal
>> > >> > > > >> [3]
>> https://github.com/apache?q=skywalking&type=all&language=&sort=
>> > >> > > > >>
>> > >> > > > >> Best,
>> > >> > > > >> tison.
>> > >> > > >
>> > >> > > > --
>> > >> > > > Xuanwo
>> > >> > > >
>> > >> > > > https://xuanwo.io/
>> > >>
>> >
>> > --
>> > Xuanwo
>> >
>> > https://xuanwo.io/
>>

-- 
Xuanwo

https://xuanwo.io/

Re: [DISCUSS] Monorepo vs. Multiple Repositories

Posted by Piotr Gankiewicz <pi...@gmail.com>.
In case of using the monorepo, could we do this in the following way:
- Move our main iggy-rs/iggy repo first under apache/iggy
- Work on structuring the new monorepo under apache/iggy to make sure we
won't break the already configured pipelines etc.
- Start moving the remaining repos under iggy-rs such as client SDKs and
other tooling to the apache/iggy monorepo once we figure out how to
structure subdirectories and the other pipelines to make it all work
- Update the readme files for all the repos under iggy-rs that all the
codebase now lives under a single monorepo at apache/iggy


On Sat, 8 Feb 2025 at 04:57, tison <wa...@gmail.com> wrote:

> Note that OpenDAL is currently releasing simulately, while each
> component can have different version.
>
> To release multilingual SDK and the main server, you should determine:
>
> 1. Version strategy (tag strategy)
> 2. How to manually/automatically deploy to the target repository
> (crates.io, Maven, PyPI, NPM etc.)
> 3. The release process to fit in ASF requirements [1][2]
>
> [1] https://www.apache.org/legal/release-policy.html
> [2] https://opendal.apache.org/community/release/
>
> Best,
> tison.
>
> Xuanwo <xu...@apache.org> 于2025年2月8日周六 11:16写道:
> >
> > Welcome to check opendal's CI:
> https://github.com/apache/opendal/tree/main/.github/workflows
> >
> > On Sat, Feb 8, 2025, at 11:15, Kranti Parisa wrote:
> > > Agreed, there are pros and cons. But I like Xuanwo's thoughts around
> > > growing the project much smoother way.
> > >
> > > Our biggest concern was around CI/CD, so if we could spend some time
> > > upfront to setup things, it should be on autopilot mode after.
> > >
> > > Are there any examples of such mono repos with different CI/CD
> > > pipelines to release SDKs and main server independently?
> > >
> > > On 2025/02/07 06:50:53 tison wrote:
> > >> The downside of mono repo is that CI/CD, especially releases, would be
> > >> much more complicated. It's possible but you need some time to work on
> > >> tools and settings.
> > >>
> > >> Best,
> > >> tison.
> > >>
> > >> tison <wa...@gmail.com> 于2025年2月7日周五 14:49写道:
> > >> >
> > >> > EDIT: [1]
> https://github.com/apache/infrastructure-asfyaml/blob/main/README.md
> > >> >
> > >> > Best,
> > >> > tison.
> > >> >
> > >> > tison <wa...@gmail.com> 于2025年2月7日周五 14:48写道:
> > >> > >
> > >> > > > First of all, after migrating the repositories to the ASF, we
> will lose admin permissions for them. This means that any operations
> requiring admin privileges will have to go through INFRA tickets unless
> they are supported by ASF tools.
> > >> > >
> > >> > > To clarify, the INFRA team is working on making the experience
> more
> > >> > > smooth, check [1][2].
> > >> > >
> > >> > > [1]
> https://github.com/apache/infrastructure-asfyaml/blob/ng-parser/docs/ASFYamlFeature.md
> > >> > > [2]
> https://github.com/apache/infrastructure-asfyaml/blob/ng-parser/docs/ASFYamlFeature.md
> > >> > >
> > >> > > But .. yes, it's not GitHub-Native.
> > >> > >
> > >> > > Best,
> > >> > > tison.
> > >> > >
> > >> > > Xuanwo <xu...@apache.org> 于2025年2月7日周五 14:45写道:
> > >> > > >
> > >> > > > I'd like to share some reasons why I prefer using a monorepo
> for OpenDAL.
> > >> > > >
> > >> > > > First of all, after migrating the repositories to the ASF, we
> will lose admin permissions for them. This means that any operations
> requiring admin privileges will have to go through INFRA tickets unless
> they are supported by ASF tools. For example, configuring GitHub secrets or
> making CI run for every PR by default. Setting this up for multiple
> repositories could be very challenging.
> > >> > > >
> > >> > > > Secondly, I find that keeping everything in the same monorepo
> makes it easier for the project to grow. Stars, issues, PRs, and
> discussions all happen in the same repository, making it easier to bring
> people together. This is especially important for a small and new project.
> We can always split the project later once we have a sufficient number of
> dedicated committers focusing on a specific area—for example, three
> committers who explicitly dedicate themselves to the Go SDK.
> > >> > > >
> > >> > > > Both setups work for me.
> > >> > > >
> > >> > > > On Fri, Feb 7, 2025, at 14:28, tison wrote:
> > >> > > > > cc in case you don't subscribe dev@ yet.
> > >> > > > >
> > >> > > > > You can reply with PonyMail also [1]
> > >> > > > >
> > >> > > > > [1]
> https://lists.apache.org/thread/zrn96nlg23r9353lr5tp2by2ggx4zxqc
> > >> > > > >
> > >> > > > > Best,
> > >> > > > > tison.
> > >> > > > >
> > >> > > > > tison <wa...@gmail.com> 于2025年2月7日周五 11:30写道:
> > >> > > > >
> > >> > > > >>
> > >> > > > >> Hi,
> > >> > > > >>
> > >> > > > >> I'd continue the discussion from [1] about whether we'd set
> up a
> > >> > > > >> monorepo for all the Iggy's code, or continue the current
> multiple
> > >> > > > >> repositories solution.
> > >> > > > >>
> > >> > > > >> [1]
> https://lists.apache.org/thread/ppqjvygk8xs7v8no5dk6r6khm5m1vp71
> > >> > > > >>
> > >> > > > >> The only thing I'd point out is that the ASF doesn't support
> > >> > > > >> fine-grained permissions so please make correct assumption.
> > >> > > > >>
> > >> > > > >> Either monorepo (OpenDAL's multilingual bindings [2]) or
> multiple
> > >> > > > >> repos (SkyWalking and its multilingual SDK [3]) works for me.
> > >> > > > >>
> > >> > > > >> [2] https://github.com/apache/opendal
> > >> > > > >> [3]
> https://github.com/apache?q=skywalking&type=all&language=&sort=
> > >> > > > >>
> > >> > > > >> Best,
> > >> > > > >> tison.
> > >> > > >
> > >> > > > --
> > >> > > > Xuanwo
> > >> > > >
> > >> > > > https://xuanwo.io/
> > >>
> >
> > --
> > Xuanwo
> >
> > https://xuanwo.io/
>

Re: [DISCUSS] Monorepo vs. Multiple Repositories

Posted by tison <wa...@gmail.com>.
Note that OpenDAL is currently releasing simulately, while each
component can have different version.

To release multilingual SDK and the main server, you should determine:

1. Version strategy (tag strategy)
2. How to manually/automatically deploy to the target repository
(crates.io, Maven, PyPI, NPM etc.)
3. The release process to fit in ASF requirements [1][2]

[1] https://www.apache.org/legal/release-policy.html
[2] https://opendal.apache.org/community/release/

Best,
tison.

Xuanwo <xu...@apache.org> 于2025年2月8日周六 11:16写道:
>
> Welcome to check opendal's CI: https://github.com/apache/opendal/tree/main/.github/workflows
>
> On Sat, Feb 8, 2025, at 11:15, Kranti Parisa wrote:
> > Agreed, there are pros and cons. But I like Xuanwo's thoughts around
> > growing the project much smoother way.
> >
> > Our biggest concern was around CI/CD, so if we could spend some time
> > upfront to setup things, it should be on autopilot mode after.
> >
> > Are there any examples of such mono repos with different CI/CD
> > pipelines to release SDKs and main server independently?
> >
> > On 2025/02/07 06:50:53 tison wrote:
> >> The downside of mono repo is that CI/CD, especially releases, would be
> >> much more complicated. It's possible but you need some time to work on
> >> tools and settings.
> >>
> >> Best,
> >> tison.
> >>
> >> tison <wa...@gmail.com> 于2025年2月7日周五 14:49写道:
> >> >
> >> > EDIT: [1] https://github.com/apache/infrastructure-asfyaml/blob/main/README.md
> >> >
> >> > Best,
> >> > tison.
> >> >
> >> > tison <wa...@gmail.com> 于2025年2月7日周五 14:48写道:
> >> > >
> >> > > > First of all, after migrating the repositories to the ASF, we will lose admin permissions for them. This means that any operations requiring admin privileges will have to go through INFRA tickets unless they are supported by ASF tools.
> >> > >
> >> > > To clarify, the INFRA team is working on making the experience more
> >> > > smooth, check [1][2].
> >> > >
> >> > > [1] https://github.com/apache/infrastructure-asfyaml/blob/ng-parser/docs/ASFYamlFeature.md
> >> > > [2] https://github.com/apache/infrastructure-asfyaml/blob/ng-parser/docs/ASFYamlFeature.md
> >> > >
> >> > > But .. yes, it's not GitHub-Native.
> >> > >
> >> > > Best,
> >> > > tison.
> >> > >
> >> > > Xuanwo <xu...@apache.org> 于2025年2月7日周五 14:45写道:
> >> > > >
> >> > > > I'd like to share some reasons why I prefer using a monorepo for OpenDAL.
> >> > > >
> >> > > > First of all, after migrating the repositories to the ASF, we will lose admin permissions for them. This means that any operations requiring admin privileges will have to go through INFRA tickets unless they are supported by ASF tools. For example, configuring GitHub secrets or making CI run for every PR by default. Setting this up for multiple repositories could be very challenging.
> >> > > >
> >> > > > Secondly, I find that keeping everything in the same monorepo makes it easier for the project to grow. Stars, issues, PRs, and discussions all happen in the same repository, making it easier to bring people together. This is especially important for a small and new project. We can always split the project later once we have a sufficient number of dedicated committers focusing on a specific area—for example, three committers who explicitly dedicate themselves to the Go SDK.
> >> > > >
> >> > > > Both setups work for me.
> >> > > >
> >> > > > On Fri, Feb 7, 2025, at 14:28, tison wrote:
> >> > > > > cc in case you don't subscribe dev@ yet.
> >> > > > >
> >> > > > > You can reply with PonyMail also [1]
> >> > > > >
> >> > > > > [1] https://lists.apache.org/thread/zrn96nlg23r9353lr5tp2by2ggx4zxqc
> >> > > > >
> >> > > > > Best,
> >> > > > > tison.
> >> > > > >
> >> > > > > tison <wa...@gmail.com> 于2025年2月7日周五 11:30写道:
> >> > > > >
> >> > > > >>
> >> > > > >> Hi,
> >> > > > >>
> >> > > > >> I'd continue the discussion from [1] about whether we'd set up a
> >> > > > >> monorepo for all the Iggy's code, or continue the current multiple
> >> > > > >> repositories solution.
> >> > > > >>
> >> > > > >> [1] https://lists.apache.org/thread/ppqjvygk8xs7v8no5dk6r6khm5m1vp71
> >> > > > >>
> >> > > > >> The only thing I'd point out is that the ASF doesn't support
> >> > > > >> fine-grained permissions so please make correct assumption.
> >> > > > >>
> >> > > > >> Either monorepo (OpenDAL's multilingual bindings [2]) or multiple
> >> > > > >> repos (SkyWalking and its multilingual SDK [3]) works for me.
> >> > > > >>
> >> > > > >> [2] https://github.com/apache/opendal
> >> > > > >> [3] https://github.com/apache?q=skywalking&type=all&language=&sort=
> >> > > > >>
> >> > > > >> Best,
> >> > > > >> tison.
> >> > > >
> >> > > > --
> >> > > > Xuanwo
> >> > > >
> >> > > > https://xuanwo.io/
> >>
>
> --
> Xuanwo
>
> https://xuanwo.io/

Re: [DISCUSS] Monorepo vs. Multiple Repositories

Posted by Xuanwo <xu...@apache.org>.
Welcome to check opendal's CI: https://github.com/apache/opendal/tree/main/.github/workflows

On Sat, Feb 8, 2025, at 11:15, Kranti Parisa wrote:
> Agreed, there are pros and cons. But I like Xuanwo's thoughts around 
> growing the project much smoother way.
>
> Our biggest concern was around CI/CD, so if we could spend some time 
> upfront to setup things, it should be on autopilot mode after.
>
> Are there any examples of such mono repos with different CI/CD 
> pipelines to release SDKs and main server independently?
>
> On 2025/02/07 06:50:53 tison wrote:
>> The downside of mono repo is that CI/CD, especially releases, would be
>> much more complicated. It's possible but you need some time to work on
>> tools and settings.
>> 
>> Best,
>> tison.
>> 
>> tison <wa...@gmail.com> 于2025年2月7日周五 14:49写道:
>> >
>> > EDIT: [1] https://github.com/apache/infrastructure-asfyaml/blob/main/README.md
>> >
>> > Best,
>> > tison.
>> >
>> > tison <wa...@gmail.com> 于2025年2月7日周五 14:48写道:
>> > >
>> > > > First of all, after migrating the repositories to the ASF, we will lose admin permissions for them. This means that any operations requiring admin privileges will have to go through INFRA tickets unless they are supported by ASF tools.
>> > >
>> > > To clarify, the INFRA team is working on making the experience more
>> > > smooth, check [1][2].
>> > >
>> > > [1] https://github.com/apache/infrastructure-asfyaml/blob/ng-parser/docs/ASFYamlFeature.md
>> > > [2] https://github.com/apache/infrastructure-asfyaml/blob/ng-parser/docs/ASFYamlFeature.md
>> > >
>> > > But .. yes, it's not GitHub-Native.
>> > >
>> > > Best,
>> > > tison.
>> > >
>> > > Xuanwo <xu...@apache.org> 于2025年2月7日周五 14:45写道:
>> > > >
>> > > > I'd like to share some reasons why I prefer using a monorepo for OpenDAL.
>> > > >
>> > > > First of all, after migrating the repositories to the ASF, we will lose admin permissions for them. This means that any operations requiring admin privileges will have to go through INFRA tickets unless they are supported by ASF tools. For example, configuring GitHub secrets or making CI run for every PR by default. Setting this up for multiple repositories could be very challenging.
>> > > >
>> > > > Secondly, I find that keeping everything in the same monorepo makes it easier for the project to grow. Stars, issues, PRs, and discussions all happen in the same repository, making it easier to bring people together. This is especially important for a small and new project. We can always split the project later once we have a sufficient number of dedicated committers focusing on a specific area—for example, three committers who explicitly dedicate themselves to the Go SDK.
>> > > >
>> > > > Both setups work for me.
>> > > >
>> > > > On Fri, Feb 7, 2025, at 14:28, tison wrote:
>> > > > > cc in case you don't subscribe dev@ yet.
>> > > > >
>> > > > > You can reply with PonyMail also [1]
>> > > > >
>> > > > > [1] https://lists.apache.org/thread/zrn96nlg23r9353lr5tp2by2ggx4zxqc
>> > > > >
>> > > > > Best,
>> > > > > tison.
>> > > > >
>> > > > > tison <wa...@gmail.com> 于2025年2月7日周五 11:30写道:
>> > > > >
>> > > > >>
>> > > > >> Hi,
>> > > > >>
>> > > > >> I'd continue the discussion from [1] about whether we'd set up a
>> > > > >> monorepo for all the Iggy's code, or continue the current multiple
>> > > > >> repositories solution.
>> > > > >>
>> > > > >> [1] https://lists.apache.org/thread/ppqjvygk8xs7v8no5dk6r6khm5m1vp71
>> > > > >>
>> > > > >> The only thing I'd point out is that the ASF doesn't support
>> > > > >> fine-grained permissions so please make correct assumption.
>> > > > >>
>> > > > >> Either monorepo (OpenDAL's multilingual bindings [2]) or multiple
>> > > > >> repos (SkyWalking and its multilingual SDK [3]) works for me.
>> > > > >>
>> > > > >> [2] https://github.com/apache/opendal
>> > > > >> [3] https://github.com/apache?q=skywalking&type=all&language=&sort=
>> > > > >>
>> > > > >> Best,
>> > > > >> tison.
>> > > >
>> > > > --
>> > > > Xuanwo
>> > > >
>> > > > https://xuanwo.io/
>>

-- 
Xuanwo

https://xuanwo.io/

Re: [DISCUSS] Monorepo vs. Multiple Repositories

Posted by Kranti Parisa <kr...@apache.org>.
Agreed, there are pros and cons. But I like Xuanwo's thoughts around growing the project much smoother way.

Our biggest concern was around CI/CD, so if we could spend some time upfront to setup things, it should be on autopilot mode after.

Are there any examples of such mono repos with different CI/CD pipelines to release SDKs and main server independently?

On 2025/02/07 06:50:53 tison wrote:
> The downside of mono repo is that CI/CD, especially releases, would be
> much more complicated. It's possible but you need some time to work on
> tools and settings.
> 
> Best,
> tison.
> 
> tison <wa...@gmail.com> 于2025年2月7日周五 14:49写道:
> >
> > EDIT: [1] https://github.com/apache/infrastructure-asfyaml/blob/main/README.md
> >
> > Best,
> > tison.
> >
> > tison <wa...@gmail.com> 于2025年2月7日周五 14:48写道:
> > >
> > > > First of all, after migrating the repositories to the ASF, we will lose admin permissions for them. This means that any operations requiring admin privileges will have to go through INFRA tickets unless they are supported by ASF tools.
> > >
> > > To clarify, the INFRA team is working on making the experience more
> > > smooth, check [1][2].
> > >
> > > [1] https://github.com/apache/infrastructure-asfyaml/blob/ng-parser/docs/ASFYamlFeature.md
> > > [2] https://github.com/apache/infrastructure-asfyaml/blob/ng-parser/docs/ASFYamlFeature.md
> > >
> > > But .. yes, it's not GitHub-Native.
> > >
> > > Best,
> > > tison.
> > >
> > > Xuanwo <xu...@apache.org> 于2025年2月7日周五 14:45写道:
> > > >
> > > > I'd like to share some reasons why I prefer using a monorepo for OpenDAL.
> > > >
> > > > First of all, after migrating the repositories to the ASF, we will lose admin permissions for them. This means that any operations requiring admin privileges will have to go through INFRA tickets unless they are supported by ASF tools. For example, configuring GitHub secrets or making CI run for every PR by default. Setting this up for multiple repositories could be very challenging.
> > > >
> > > > Secondly, I find that keeping everything in the same monorepo makes it easier for the project to grow. Stars, issues, PRs, and discussions all happen in the same repository, making it easier to bring people together. This is especially important for a small and new project. We can always split the project later once we have a sufficient number of dedicated committers focusing on a specific area—for example, three committers who explicitly dedicate themselves to the Go SDK.
> > > >
> > > > Both setups work for me.
> > > >
> > > > On Fri, Feb 7, 2025, at 14:28, tison wrote:
> > > > > cc in case you don't subscribe dev@ yet.
> > > > >
> > > > > You can reply with PonyMail also [1]
> > > > >
> > > > > [1] https://lists.apache.org/thread/zrn96nlg23r9353lr5tp2by2ggx4zxqc
> > > > >
> > > > > Best,
> > > > > tison.
> > > > >
> > > > > tison <wa...@gmail.com> 于2025年2月7日周五 11:30写道:
> > > > >
> > > > >>
> > > > >> Hi,
> > > > >>
> > > > >> I'd continue the discussion from [1] about whether we'd set up a
> > > > >> monorepo for all the Iggy's code, or continue the current multiple
> > > > >> repositories solution.
> > > > >>
> > > > >> [1] https://lists.apache.org/thread/ppqjvygk8xs7v8no5dk6r6khm5m1vp71
> > > > >>
> > > > >> The only thing I'd point out is that the ASF doesn't support
> > > > >> fine-grained permissions so please make correct assumption.
> > > > >>
> > > > >> Either monorepo (OpenDAL's multilingual bindings [2]) or multiple
> > > > >> repos (SkyWalking and its multilingual SDK [3]) works for me.
> > > > >>
> > > > >> [2] https://github.com/apache/opendal
> > > > >> [3] https://github.com/apache?q=skywalking&type=all&language=&sort=
> > > > >>
> > > > >> Best,
> > > > >> tison.
> > > >
> > > > --
> > > > Xuanwo
> > > >
> > > > https://xuanwo.io/
> 

Re: [DISCUSS] Monorepo vs. Multiple Repositories

Posted by tison <wa...@gmail.com>.
The downside of mono repo is that CI/CD, especially releases, would be
much more complicated. It's possible but you need some time to work on
tools and settings.

Best,
tison.

tison <wa...@gmail.com> 于2025年2月7日周五 14:49写道:
>
> EDIT: [1] https://github.com/apache/infrastructure-asfyaml/blob/main/README.md
>
> Best,
> tison.
>
> tison <wa...@gmail.com> 于2025年2月7日周五 14:48写道:
> >
> > > First of all, after migrating the repositories to the ASF, we will lose admin permissions for them. This means that any operations requiring admin privileges will have to go through INFRA tickets unless they are supported by ASF tools.
> >
> > To clarify, the INFRA team is working on making the experience more
> > smooth, check [1][2].
> >
> > [1] https://github.com/apache/infrastructure-asfyaml/blob/ng-parser/docs/ASFYamlFeature.md
> > [2] https://github.com/apache/infrastructure-asfyaml/blob/ng-parser/docs/ASFYamlFeature.md
> >
> > But .. yes, it's not GitHub-Native.
> >
> > Best,
> > tison.
> >
> > Xuanwo <xu...@apache.org> 于2025年2月7日周五 14:45写道:
> > >
> > > I'd like to share some reasons why I prefer using a monorepo for OpenDAL.
> > >
> > > First of all, after migrating the repositories to the ASF, we will lose admin permissions for them. This means that any operations requiring admin privileges will have to go through INFRA tickets unless they are supported by ASF tools. For example, configuring GitHub secrets or making CI run for every PR by default. Setting this up for multiple repositories could be very challenging.
> > >
> > > Secondly, I find that keeping everything in the same monorepo makes it easier for the project to grow. Stars, issues, PRs, and discussions all happen in the same repository, making it easier to bring people together. This is especially important for a small and new project. We can always split the project later once we have a sufficient number of dedicated committers focusing on a specific area—for example, three committers who explicitly dedicate themselves to the Go SDK.
> > >
> > > Both setups work for me.
> > >
> > > On Fri, Feb 7, 2025, at 14:28, tison wrote:
> > > > cc in case you don't subscribe dev@ yet.
> > > >
> > > > You can reply with PonyMail also [1]
> > > >
> > > > [1] https://lists.apache.org/thread/zrn96nlg23r9353lr5tp2by2ggx4zxqc
> > > >
> > > > Best,
> > > > tison.
> > > >
> > > > tison <wa...@gmail.com> 于2025年2月7日周五 11:30写道:
> > > >
> > > >>
> > > >> Hi,
> > > >>
> > > >> I'd continue the discussion from [1] about whether we'd set up a
> > > >> monorepo for all the Iggy's code, or continue the current multiple
> > > >> repositories solution.
> > > >>
> > > >> [1] https://lists.apache.org/thread/ppqjvygk8xs7v8no5dk6r6khm5m1vp71
> > > >>
> > > >> The only thing I'd point out is that the ASF doesn't support
> > > >> fine-grained permissions so please make correct assumption.
> > > >>
> > > >> Either monorepo (OpenDAL's multilingual bindings [2]) or multiple
> > > >> repos (SkyWalking and its multilingual SDK [3]) works for me.
> > > >>
> > > >> [2] https://github.com/apache/opendal
> > > >> [3] https://github.com/apache?q=skywalking&type=all&language=&sort=
> > > >>
> > > >> Best,
> > > >> tison.
> > >
> > > --
> > > Xuanwo
> > >
> > > https://xuanwo.io/

Re: [DISCUSS] Monorepo vs. Multiple Repositories

Posted by tison <wa...@gmail.com>.
EDIT: [1] https://github.com/apache/infrastructure-asfyaml/blob/main/README.md

Best,
tison.

tison <wa...@gmail.com> 于2025年2月7日周五 14:48写道:
>
> > First of all, after migrating the repositories to the ASF, we will lose admin permissions for them. This means that any operations requiring admin privileges will have to go through INFRA tickets unless they are supported by ASF tools.
>
> To clarify, the INFRA team is working on making the experience more
> smooth, check [1][2].
>
> [1] https://github.com/apache/infrastructure-asfyaml/blob/ng-parser/docs/ASFYamlFeature.md
> [2] https://github.com/apache/infrastructure-asfyaml/blob/ng-parser/docs/ASFYamlFeature.md
>
> But .. yes, it's not GitHub-Native.
>
> Best,
> tison.
>
> Xuanwo <xu...@apache.org> 于2025年2月7日周五 14:45写道:
> >
> > I'd like to share some reasons why I prefer using a monorepo for OpenDAL.
> >
> > First of all, after migrating the repositories to the ASF, we will lose admin permissions for them. This means that any operations requiring admin privileges will have to go through INFRA tickets unless they are supported by ASF tools. For example, configuring GitHub secrets or making CI run for every PR by default. Setting this up for multiple repositories could be very challenging.
> >
> > Secondly, I find that keeping everything in the same monorepo makes it easier for the project to grow. Stars, issues, PRs, and discussions all happen in the same repository, making it easier to bring people together. This is especially important for a small and new project. We can always split the project later once we have a sufficient number of dedicated committers focusing on a specific area—for example, three committers who explicitly dedicate themselves to the Go SDK.
> >
> > Both setups work for me.
> >
> > On Fri, Feb 7, 2025, at 14:28, tison wrote:
> > > cc in case you don't subscribe dev@ yet.
> > >
> > > You can reply with PonyMail also [1]
> > >
> > > [1] https://lists.apache.org/thread/zrn96nlg23r9353lr5tp2by2ggx4zxqc
> > >
> > > Best,
> > > tison.
> > >
> > > tison <wa...@gmail.com> 于2025年2月7日周五 11:30写道:
> > >
> > >>
> > >> Hi,
> > >>
> > >> I'd continue the discussion from [1] about whether we'd set up a
> > >> monorepo for all the Iggy's code, or continue the current multiple
> > >> repositories solution.
> > >>
> > >> [1] https://lists.apache.org/thread/ppqjvygk8xs7v8no5dk6r6khm5m1vp71
> > >>
> > >> The only thing I'd point out is that the ASF doesn't support
> > >> fine-grained permissions so please make correct assumption.
> > >>
> > >> Either monorepo (OpenDAL's multilingual bindings [2]) or multiple
> > >> repos (SkyWalking and its multilingual SDK [3]) works for me.
> > >>
> > >> [2] https://github.com/apache/opendal
> > >> [3] https://github.com/apache?q=skywalking&type=all&language=&sort=
> > >>
> > >> Best,
> > >> tison.
> >
> > --
> > Xuanwo
> >
> > https://xuanwo.io/

Re: [DISCUSS] Monorepo vs. Multiple Repositories

Posted by tison <wa...@gmail.com>.
> First of all, after migrating the repositories to the ASF, we will lose admin permissions for them. This means that any operations requiring admin privileges will have to go through INFRA tickets unless they are supported by ASF tools.

To clarify, the INFRA team is working on making the experience more
smooth, check [1][2].

[1] https://github.com/apache/infrastructure-asfyaml/blob/ng-parser/docs/ASFYamlFeature.md
[2] https://github.com/apache/infrastructure-asfyaml/blob/ng-parser/docs/ASFYamlFeature.md

But .. yes, it's not GitHub-Native.

Best,
tison.

Xuanwo <xu...@apache.org> 于2025年2月7日周五 14:45写道:
>
> I'd like to share some reasons why I prefer using a monorepo for OpenDAL.
>
> First of all, after migrating the repositories to the ASF, we will lose admin permissions for them. This means that any operations requiring admin privileges will have to go through INFRA tickets unless they are supported by ASF tools. For example, configuring GitHub secrets or making CI run for every PR by default. Setting this up for multiple repositories could be very challenging.
>
> Secondly, I find that keeping everything in the same monorepo makes it easier for the project to grow. Stars, issues, PRs, and discussions all happen in the same repository, making it easier to bring people together. This is especially important for a small and new project. We can always split the project later once we have a sufficient number of dedicated committers focusing on a specific area—for example, three committers who explicitly dedicate themselves to the Go SDK.
>
> Both setups work for me.
>
> On Fri, Feb 7, 2025, at 14:28, tison wrote:
> > cc in case you don't subscribe dev@ yet.
> >
> > You can reply with PonyMail also [1]
> >
> > [1] https://lists.apache.org/thread/zrn96nlg23r9353lr5tp2by2ggx4zxqc
> >
> > Best,
> > tison.
> >
> > tison <wa...@gmail.com> 于2025年2月7日周五 11:30写道:
> >
> >>
> >> Hi,
> >>
> >> I'd continue the discussion from [1] about whether we'd set up a
> >> monorepo for all the Iggy's code, or continue the current multiple
> >> repositories solution.
> >>
> >> [1] https://lists.apache.org/thread/ppqjvygk8xs7v8no5dk6r6khm5m1vp71
> >>
> >> The only thing I'd point out is that the ASF doesn't support
> >> fine-grained permissions so please make correct assumption.
> >>
> >> Either monorepo (OpenDAL's multilingual bindings [2]) or multiple
> >> repos (SkyWalking and its multilingual SDK [3]) works for me.
> >>
> >> [2] https://github.com/apache/opendal
> >> [3] https://github.com/apache?q=skywalking&type=all&language=&sort=
> >>
> >> Best,
> >> tison.
>
> --
> Xuanwo
>
> https://xuanwo.io/

Re: [DISCUSS] Monorepo vs. Multiple Repositories

Posted by Xuanwo <xu...@apache.org>.
I'd like to share some reasons why I prefer using a monorepo for OpenDAL.

First of all, after migrating the repositories to the ASF, we will lose admin permissions for them. This means that any operations requiring admin privileges will have to go through INFRA tickets unless they are supported by ASF tools. For example, configuring GitHub secrets or making CI run for every PR by default. Setting this up for multiple repositories could be very challenging.

Secondly, I find that keeping everything in the same monorepo makes it easier for the project to grow. Stars, issues, PRs, and discussions all happen in the same repository, making it easier to bring people together. This is especially important for a small and new project. We can always split the project later once we have a sufficient number of dedicated committers focusing on a specific area—for example, three committers who explicitly dedicate themselves to the Go SDK.

Both setups work for me.

On Fri, Feb 7, 2025, at 14:28, tison wrote:
> cc in case you don't subscribe dev@ yet.
>
> You can reply with PonyMail also [1]
>
> [1] https://lists.apache.org/thread/zrn96nlg23r9353lr5tp2by2ggx4zxqc
>
> Best,
> tison.
>
> tison <wa...@gmail.com> 于2025年2月7日周五 11:30写道:
>
>>
>> Hi,
>>
>> I'd continue the discussion from [1] about whether we'd set up a
>> monorepo for all the Iggy's code, or continue the current multiple
>> repositories solution.
>>
>> [1] https://lists.apache.org/thread/ppqjvygk8xs7v8no5dk6r6khm5m1vp71
>>
>> The only thing I'd point out is that the ASF doesn't support
>> fine-grained permissions so please make correct assumption.
>>
>> Either monorepo (OpenDAL's multilingual bindings [2]) or multiple
>> repos (SkyWalking and its multilingual SDK [3]) works for me.
>>
>> [2] https://github.com/apache/opendal
>> [3] https://github.com/apache?q=skywalking&type=all&language=&sort=
>>
>> Best,
>> tison.

-- 
Xuanwo

https://xuanwo.io/

Re: [DISCUSS] Monorepo vs. Multiple Repositories

Posted by tison <wa...@gmail.com>.
cc in case you don't subscribe dev@ yet.

You can reply with PonyMail also [1]

[1] https://lists.apache.org/thread/zrn96nlg23r9353lr5tp2by2ggx4zxqc

Best,
tison.

tison <wa...@gmail.com> 于2025年2月7日周五 11:30写道:

>
> Hi,
>
> I'd continue the discussion from [1] about whether we'd set up a
> monorepo for all the Iggy's code, or continue the current multiple
> repositories solution.
>
> [1] https://lists.apache.org/thread/ppqjvygk8xs7v8no5dk6r6khm5m1vp71
>
> The only thing I'd point out is that the ASF doesn't support
> fine-grained permissions so please make correct assumption.
>
> Either monorepo (OpenDAL's multilingual bindings [2]) or multiple
> repos (SkyWalking and its multilingual SDK [3]) works for me.
>
> [2] https://github.com/apache/opendal
> [3] https://github.com/apache?q=skywalking&type=all&language=&sort=
>
> Best,
> tison.