You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Sutou Kouhei <ko...@clear-code.com> on 2024/07/18 09:33:04 UTC
[DISCUSS] Split Go release process
Hi,
This is a continuation of the "[DISCUSS] Versioning and
releases for apache/arrow components" thread:
https://lists.apache.org/thread/tvqns9d9z45mtpsqrngjyg083jdv8f1t
How about splitting Go release process from other
apache/arrow components as the first try? Here are reasons
why Go is the best candidate:
* Go doesn't depend on other components such as C++
* Go has some active PMC member (Matt) and committer (Joel)
* Could you become a release manager for Go?
Here is my idea how to proceed this:
1. Extract go/ in apache/arrow to apache/arrow-go like
apache/arrow-rs
* Filter go/ related commits from apache/arrow and create
apache/arrow-go with them like we did for apache/arrow-rs
* Remove go/ related codes from apache/arrow
2. Prepare integration test CI like apache/arrow-rs does:
https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml
3. Prepare release script based on apache/arrow-julia,
apache/arrow-adbc and/or apache/arrow-flight-sql-postgresql
We can proceed this in apache/arrow but it may be difficult
to maintain and release a new version. The apache/arrow-go
approach needs more work at the initial step but we will be
able to release easily after that like apache/arrow-julia.
We can reuse some release scripts in dev/release/ in
apache/arrow like we did in apache/arrow-adbc.
Cons of this idea:
* This is a backward incompatible change
* Users need to change their "import" to
"github.com/apache/arrow-go/arrow" from
"github.com/apache/arrow/go/arrow"
What do you think about this?
Thanks,
--
kou
Re: [DISCUSS] Split Go release process
Posted by Sutou Kouhei <ko...@clear-code.com>.
Hi,
> The import path could be "github.com/apache/arrow-go" instead of "github.com/apache/arrow-go/arrow". Since go will allow users to use `arrow.Abc` directly if user imports `github.com/apache/arrow-go`.
Thanks for the suggestion.
I didn't know about it. I'm not familiar with Go's import
system...
Thanks,
--
kou
In <5c...@app.fastmail.com>
"Re: [DISCUSS] Split Go release process" on Thu, 18 Jul 2024 17:38:55 +0800,
Xuanwo <xu...@apache.org> wrote:
> Hi, Sutou Kouhei
>
> The process looks good to me, and I think it's beneficial for a Go project to have a separate repository.
>
> I only have one suggestion:
>
> The import path could be "github.com/apache/arrow-go" instead of "github.com/apache/arrow-go/arrow". Since go will allow users to use `arrow.Abc` directly if user imports `github.com/apache/arrow-go`.
>
> On Thu, Jul 18, 2024, at 17:33, Sutou Kouhei wrote:
>> Hi,
>>
>> This is a continuation of the "[DISCUSS] Versioning and
>> releases for apache/arrow components" thread:
>> https://lists.apache.org/thread/tvqns9d9z45mtpsqrngjyg083jdv8f1t
>>
>> How about splitting Go release process from other
>> apache/arrow components as the first try? Here are reasons
>> why Go is the best candidate:
>>
>> * Go doesn't depend on other components such as C++
>> * Go has some active PMC member (Matt) and committer (Joel)
>> * Could you become a release manager for Go?
>>
>> Here is my idea how to proceed this:
>>
>> 1. Extract go/ in apache/arrow to apache/arrow-go like
>> apache/arrow-rs
>> * Filter go/ related commits from apache/arrow and create
>> apache/arrow-go with them like we did for apache/arrow-rs
>> * Remove go/ related codes from apache/arrow
>> 2. Prepare integration test CI like apache/arrow-rs does:
>>
>> https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml
>> 3. Prepare release script based on apache/arrow-julia,
>> apache/arrow-adbc and/or apache/arrow-flight-sql-postgresql
>>
>> We can proceed this in apache/arrow but it may be difficult
>> to maintain and release a new version. The apache/arrow-go
>> approach needs more work at the initial step but we will be
>> able to release easily after that like apache/arrow-julia.
>>
>> We can reuse some release scripts in dev/release/ in
>> apache/arrow like we did in apache/arrow-adbc.
>>
>> Cons of this idea:
>>
>> * This is a backward incompatible change
>> * Users need to change their "import" to
>> "github.com/apache/arrow-go/arrow" from
>> "github.com/apache/arrow/go/arrow"
>>
>>
>> What do you think about this?
>>
>>
>> Thanks,
>> --
>> kou
>
> --
> Xuanwo
>
> https://xuanwo.io/
Re: [DISCUSS] Split Go release process
Posted by Xuanwo <xu...@apache.org>.
Hi, Sutou Kouhei
The process looks good to me, and I think it's beneficial for a Go project to have a separate repository.
I only have one suggestion:
The import path could be "github.com/apache/arrow-go" instead of "github.com/apache/arrow-go/arrow". Since go will allow users to use `arrow.Abc` directly if user imports `github.com/apache/arrow-go`.
On Thu, Jul 18, 2024, at 17:33, Sutou Kouhei wrote:
> Hi,
>
> This is a continuation of the "[DISCUSS] Versioning and
> releases for apache/arrow components" thread:
> https://lists.apache.org/thread/tvqns9d9z45mtpsqrngjyg083jdv8f1t
>
> How about splitting Go release process from other
> apache/arrow components as the first try? Here are reasons
> why Go is the best candidate:
>
> * Go doesn't depend on other components such as C++
> * Go has some active PMC member (Matt) and committer (Joel)
> * Could you become a release manager for Go?
>
> Here is my idea how to proceed this:
>
> 1. Extract go/ in apache/arrow to apache/arrow-go like
> apache/arrow-rs
> * Filter go/ related commits from apache/arrow and create
> apache/arrow-go with them like we did for apache/arrow-rs
> * Remove go/ related codes from apache/arrow
> 2. Prepare integration test CI like apache/arrow-rs does:
>
> https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml
> 3. Prepare release script based on apache/arrow-julia,
> apache/arrow-adbc and/or apache/arrow-flight-sql-postgresql
>
> We can proceed this in apache/arrow but it may be difficult
> to maintain and release a new version. The apache/arrow-go
> approach needs more work at the initial step but we will be
> able to release easily after that like apache/arrow-julia.
>
> We can reuse some release scripts in dev/release/ in
> apache/arrow like we did in apache/arrow-adbc.
>
> Cons of this idea:
>
> * This is a backward incompatible change
> * Users need to change their "import" to
> "github.com/apache/arrow-go/arrow" from
> "github.com/apache/arrow/go/arrow"
>
>
> What do you think about this?
>
>
> Thanks,
> --
> kou
--
Xuanwo
https://xuanwo.io/
Re: [DISCUSS] Split Go release process
Posted by Sutou Kouhei <ko...@clear-code.com>.
Hi,
> Is there no way to leave some kind of alias or redirection in the
> apache/arrow repository?
I'm looking at this in https://github.com/kou/arrow-go-alias
but I haven't found a way for it yet. If anyone has an idea
how to implement it, could you share your idea?
Thanks,
--
kou
In <32...@python.org>
"Re: [DISCUSS] Split Go release process" on Thu, 18 Jul 2024 13:32:57 +0200,
Antoine Pitrou <an...@python.org> wrote:
>
> Hi Kou,
>
> Le 18/07/2024 à 11:33, Sutou Kouhei a écrit :
>> Here is my idea how to proceed this:
>> 1. Extract go/ in apache/arrow to apache/arrow-go like
>> apache/arrow-rs
>> * Filter go/ related commits from apache/arrow and create
>> apache/arrow-go with them like we did for apache/arrow-rs
>> * Remove go/ related codes from apache/arrow
>> 2. Prepare integration test CI like apache/arrow-rs does:
>> https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml
>> 3. Prepare release script based on apache/arrow-julia,
>> apache/arrow-adbc and/or apache/arrow-flight-sql-postgresql
>
> I think this is a good idea, but I'm not part of the Go maintainers.
>
>> Cons of this idea:
>> * This is a backward incompatible change
>> * Users need to change their "import" to
>> "github.com/apache/arrow-go/arrow" from
>> "github.com/apache/arrow/go/arrow"
>
> Is there no way to leave some kind of alias or redirection in the
> apache/arrow repository?
>
> Regards
>
> Antoine.
Re: [DISCUSS] Split Go release process
Posted by Sutou Kouhei <ko...@clear-code.com>.
Hi,
> My vote - for whatever it's worth - would be to do away with the
> version-in-path naming convention and relying on the go version/package
> system for major upgrades.
Could you share the document URL of the "go version/package
system"? I'm not familiar with it.
Thanks,
--
kou
In <CA...@mail.gmail.com>
"Re: [DISCUSS] Split Go release process" on Thu, 18 Jul 2024 15:54:33 -0400,
George Godik <gg...@gmail.com> wrote:
>> If we shift the Go lib to a new/different import
> path we'll end up with the same problem where people will rely on older
> versions and an incorrect path.
>
> Major version upgrades already require changing the import paths by
> increasing the version. The proposed change would require everyone to go
> through a similar process one last time.
>
>> More to the point, there would be the question of whether or not we
> should port over the same major version
> number, i.e. `github.com/apache/arrow-go/v17`
> <http://github.com/apache/arrow-go/v17> or something to that end? Or
> do we restart back at v1 (which I think would be confusing)?
>
> My vote - for whatever it's worth - would be to do away with the
> version-in-path naming convention and relying on the go version/package
> system for major upgrades.
>
> Benefits: I don't have to change import paths every 3-6months
>
> On Thu, Jul 18, 2024 at 3:34 PM Matt Topol <zo...@gmail.com> wrote:
>
>> My thoughts:
>>
>> > * Go doesn't depend on other components such as C++
>> > * Go has some active PMC member (Matt) and committer (Joel)
>> > * Could you become a release manager for Go?
>>
>> I'd happily be the release manager for the Go implementation.
>>
>> > Here is my idea how to proceed this:
>> >
>> > 1. Extract go/ in apache/arrow to apache/arrow-go like
>> > apache/arrow-rs
>> > * Filter go/ related commits from apache/arrow and create
>> > apache/arrow-go with them like we did for apache/arrow-rs
>> > * Remove go/ related codes from apache/arrow
>> > 2. Prepare integration test CI like apache/arrow-rs does:
>> >
>>
>> https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml
>> > 3. Prepare release script based on apache/arrow-julia,
>> > apache/arrow-adbc and/or apache/arrow-flight-sql-postgresql
>>
>> Personally I would prefer that we do not extract it to its own separate
>> repository purely because I don't want to change the import path for users
>> again. We already have this issue from before we introduced the major
>> version into the import path and shifted it up to allow for the Parquet lib
>> in the same repository. If you look at [1] you see that there's still over
>> 100 projects that never upgraded to v6 or higher because they are still
>> using the old import path. If we shift the Go lib to a new/different import
>> path we'll end up with the same problem where people will rely on older
>> versions and an incorrect path.
>>
>> If we as a community decide that splitting out the implementations all into
>> separate repositories is the best way forward, I won't hold it up by
>> strictly hammering on this. I'm just concerned about the realities and
>> difficulties of communicating the import path change, ensuring we don't
>> break any consumers, and ensuring that users still end up being able to
>> upgrade easily.
>>
>> > The import path could be "github.com/apache/arrow-go" instead of "
>> github.com/apache/arrow-go/arrow". Since go will allow users to use
>> `arrow.Abc` directly if user imports `github.com/apache/arrow-go`
>> <http://github.com/apache/arrow-go>
>> <http://github.com/apache/arrow-go>.
>>
>> The import path would still have to be `github.com/apache/arrow-go/arrow`
>> <http://github.com/apache/arrow-go/arrow>
>> since it would also contain the parquet implementation in `
>> github.com/apache/arrow-go/parquet`
>> <http://github.com/apache/arrow-go/parquet>. More to the point, there
>> would be the
>> question of whether or not we should port over the same major version
>> number, i.e. `github.com/apache/arrow-go/v17`
>> <http://github.com/apache/arrow-go/v17> or something to that end? Or
>> do we restart back at v1 (which I think would be confusing)?
>>
>> --Matt
>>
>> [1]: https://pkg.go.dev/github.com/apache/arrow/go/arrow
>>
>> On Thu, Jul 18, 2024 at 7:33 AM Antoine Pitrou <an...@python.org> wrote:
>>
>> >
>> > Hi Kou,
>> >
>> > Le 18/07/2024 à 11:33, Sutou Kouhei a écrit :
>> > >
>> > > Here is my idea how to proceed this:
>> > >
>> > > 1. Extract go/ in apache/arrow to apache/arrow-go like
>> > > apache/arrow-rs
>> > > * Filter go/ related commits from apache/arrow and create
>> > > apache/arrow-go with them like we did for apache/arrow-rs
>> > > * Remove go/ related codes from apache/arrow
>> > > 2. Prepare integration test CI like apache/arrow-rs does:
>> > >
>> >
>> https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml
>> > > 3. Prepare release script based on apache/arrow-julia,
>> > > apache/arrow-adbc and/or apache/arrow-flight-sql-postgresql
>> >
>> > I think this is a good idea, but I'm not part of the Go maintainers.
>> >
>> > > Cons of this idea:
>> > >
>> > > * This is a backward incompatible change
>> > > * Users need to change their "import" to
>> > > "github.com/apache/arrow-go/arrow" from
>> > > "github.com/apache/arrow/go/arrow"
>> >
>> > Is there no way to leave some kind of alias or redirection in the
>> > apache/arrow repository?
>> >
>> > Regards
>> >
>> > Antoine.
>> >
>>
Re: [DISCUSS] Split Go release process
Posted by Sutou Kouhei <ko...@clear-code.com>.
Hi,
> I don't want to change the import path for users
> again.
I agree with you. It will be very inconvenient for users.
Do you know a way to alias the import path?
> More to the point, there would be the
> question of whether or not we should port over the same major version
> number, i.e. `github.com/apache/arrow-go/v17` or something to that end? Or
> do we restart back at v1 (which I think would be confusing)?
I think that we should port over the same major version
number. github.com/apache/arrow-go/v16 and so on should be
supported too for easy migration.
Thanks,
--
kou
In <CA...@mail.gmail.com>
"Re: [DISCUSS] Split Go release process" on Thu, 18 Jul 2024 15:26:09 -0400,
Matt Topol <zo...@gmail.com> wrote:
> My thoughts:
>
>> * Go doesn't depend on other components such as C++
>> * Go has some active PMC member (Matt) and committer (Joel)
>> * Could you become a release manager for Go?
>
> I'd happily be the release manager for the Go implementation.
>
>> Here is my idea how to proceed this:
>>
>> 1. Extract go/ in apache/arrow to apache/arrow-go like
>> apache/arrow-rs
>> * Filter go/ related commits from apache/arrow and create
>> apache/arrow-go with them like we did for apache/arrow-rs
>> * Remove go/ related codes from apache/arrow
>> 2. Prepare integration test CI like apache/arrow-rs does:
>>
> https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml
>> 3. Prepare release script based on apache/arrow-julia,
>> apache/arrow-adbc and/or apache/arrow-flight-sql-postgresql
>
> Personally I would prefer that we do not extract it to its own separate
> repository purely because I don't want to change the import path for users
> again. We already have this issue from before we introduced the major
> version into the import path and shifted it up to allow for the Parquet lib
> in the same repository. If you look at [1] you see that there's still over
> 100 projects that never upgraded to v6 or higher because they are still
> using the old import path. If we shift the Go lib to a new/different import
> path we'll end up with the same problem where people will rely on older
> versions and an incorrect path.
>
> If we as a community decide that splitting out the implementations all into
> separate repositories is the best way forward, I won't hold it up by
> strictly hammering on this. I'm just concerned about the realities and
> difficulties of communicating the import path change, ensuring we don't
> break any consumers, and ensuring that users still end up being able to
> upgrade easily.
>
>> The import path could be "github.com/apache/arrow-go" instead of "
> github.com/apache/arrow-go/arrow". Since go will allow users to use
> `arrow.Abc` directly if user imports `github.com/apache/arrow-go`
> <http://github.com/apache/arrow-go>.
>
> The import path would still have to be `github.com/apache/arrow-go/arrow`
> since it would also contain the parquet implementation in `
> github.com/apache/arrow-go/parquet`. More to the point, there would be the
> question of whether or not we should port over the same major version
> number, i.e. `github.com/apache/arrow-go/v17` or something to that end? Or
> do we restart back at v1 (which I think would be confusing)?
>
> --Matt
>
> [1]: https://pkg.go.dev/github.com/apache/arrow/go/arrow
>
> On Thu, Jul 18, 2024 at 7:33 AM Antoine Pitrou <an...@python.org> wrote:
>
>>
>> Hi Kou,
>>
>> Le 18/07/2024 à 11:33, Sutou Kouhei a écrit :
>> >
>> > Here is my idea how to proceed this:
>> >
>> > 1. Extract go/ in apache/arrow to apache/arrow-go like
>> > apache/arrow-rs
>> > * Filter go/ related commits from apache/arrow and create
>> > apache/arrow-go with them like we did for apache/arrow-rs
>> > * Remove go/ related codes from apache/arrow
>> > 2. Prepare integration test CI like apache/arrow-rs does:
>> >
>> https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml
>> > 3. Prepare release script based on apache/arrow-julia,
>> > apache/arrow-adbc and/or apache/arrow-flight-sql-postgresql
>>
>> I think this is a good idea, but I'm not part of the Go maintainers.
>>
>> > Cons of this idea:
>> >
>> > * This is a backward incompatible change
>> > * Users need to change their "import" to
>> > "github.com/apache/arrow-go/arrow" from
>> > "github.com/apache/arrow/go/arrow"
>>
>> Is there no way to leave some kind of alias or redirection in the
>> apache/arrow repository?
>>
>> Regards
>>
>> Antoine.
>>
Re: [DISCUSS] Split Go release process
Posted by Sutou Kouhei <ko...@clear-code.com>.
Hi,
No objection. I'll start a vote for this.
Thanks,
--
kou
In <20...@clear-code.com>
"Re: [DISCUSS] Split Go release process" on Tue, 20 Aug 2024 09:17:45 +0900 (JST),
Sutou Kouhei <ko...@clear-code.com> wrote:
> Hi,
>
>> I think we'll need a vote on it *eventually*, but we probably can wait
>> until the repo is up and ready with the vote being when we "pull the
>> trigger" so-to-say and turn off Go releases from the main
>> github.com/apache/arrow repository and start releasing through
>> github.com/apache/arrow-go. We'll also probably need to shift open PRs /
>> issues to the new repo after we do this.
>
> OK. I'll start a vote next week if nobody objects this
> proposal in this week. I think that a vote after preparing
> github.com/apache/arrow-go may inhibit -1 vote. So a vote
> before preparing github.com/apache/arrow-go will be fair.
>
>
> Thanks,
> --
> kou
>
> In <CA...@mail.gmail.com>
> "Re: [DISCUSS] Split Go release process" on Mon, 19 Aug 2024 16:15:52 -0400,
> Matt Topol <zo...@gmail.com> wrote:
>
>>> Based on 6., users need to change their import paths on
>>> upgrade whether we keep using apache/arrow or we use new
>>> apache/arrow-go.
>>>
>>> If we use new apache/arrow-go, we will be able to reduce
>>> maintenance cost for apache/arrow (e.g. we can remove Go
>>> related scripts, CI jobs and so on from apache/arrow). Let's
>>> use apache/arrow-go.
>>>
>>> If nobody objects splitting Apache Arrow Go to
>>> apache/arrow-go in this week, I'll start working on this
>>> next week. (Do we need a vote for this?)
>>
>> I think we'll need a vote on it *eventually*, but we probably can wait
>> until the repo is up and ready with the vote being when we "pull the
>> trigger" so-to-say and turn off Go releases from the main
>> github.com/apache/arrow repository and start releasing through
>> github.com/apache/arrow-go. We'll also probably need to shift open PRs /
>> issues to the new repo after we do this.
>>
>> I'm happy to review any of the scripts being added to the new repository
>> and help out if you need it in getting this going. I think the prospect of
>> fewer major releases, and thus easier upgrades, is very worthwhile for us
>> to pursue this.
>>
>> --Matt
>>
>> On Mon, Aug 19, 2024 at 1:14 AM Sutou Kouhei <ko...@clear-code.com> wrote:
>>
>>> Hi,
>>>
>>> Sorry for not working on this.
>>>
>>> Thanks for sharing the standard docs! I've read it and
>>> related docs.
>>>
>>> Here is the summary I learned in this thread and the
>>> standard docs:
>>>
>>> 1. We're using "github.com/apache/arrow/go/v${VERSION}
>>> <http://github.com/apache/arrow/go/v$%7BVERSION%7D>" such
>>> as "github.com/apache/arrow/go/v17" as our module name
>>> * https://pkg.go.dev/github.com/apache/arrow/go/v17/arrow
>>> * Including the version number part ("v${VERSION}") is
>>> important
>>> * Users can avoid unexpected backward incompatibility by
>>> this style
>>> 2. We used to use "github.com/apache/arrow/go" as our module
>>> name in v5 or earlier
>>> * https://pkg.go.dev/github.com/apache/arrow/go/arrow
>>> * 133 modules still use this
>>> 3. We want to avoid user side changes as much as possible
>>> * As 2. shows, users may keep using old version if there
>>> is any change is required
>>> 4. The current users need to change Apache Arrow Go's import
>>> path to "github.com/apache/arrow/go/v${VERSION
>>> <http://github.com/apache/arrow/go/v$%7BVERSION> + 1}" when
>>> they want to upgrade Apache Arrow Go
>>> * We don't want to require more changes than "changing
>>> import path" for users as mentioned in 3.
>>> 5. We can't provide backward compatible module name such as
>>> "github.com/apache/arrow/go/v18" for
>>> "github.com/apache/arrow-go/v18"
>>> * Go doesn't provide the feature
>>> 6. We want to keep "v${VERSION}" in our module name even if
>>> we split Apache Arrow Go to apache/arrow-go
>>> * It's for avoiding unexpected backward incompatibility
>>> in users' projects
>>>
>>>
>>> Based on 6., users need to change their import paths on
>>> upgrade whether we keep using apache/arrow or we use new
>>> apache/arrow-go.
>>>
>>> If we use new apache/arrow-go, we will be able to reduce
>>> maintenance cost for apache/arrow (e.g. we can remove Go
>>> related scripts, CI jobs and so on from apache/arrow). Let's
>>> use apache/arrow-go.
>>>
>>> If nobody objects splitting Apache Arrow Go to
>>> apache/arrow-go in this week, I'll start working on this
>>> next week. (Do we need a vote for this?)
>>>
>>>
>>> Thanks,
>>> --
>>> kou
>>>
>>> In <CA...@mail.gmail.com>
>>> "Re: [DISCUSS] Split Go release process" on Mon, 22 Jul 2024 20:47:57
>>> -0400,
>>> Matt Topol <zo...@gmail.com> wrote:
>>>
>>> > Hey Kou,
>>> >
>>> > https://go.dev/doc/modules/release-workflow is the standard docs for
>>> > developing module versioning and publishing with Go.
>>> >
>>> > There isn't really a way to alias an import path to a different git repo
>>> > because it uses the GitHub URL itself as the import path.
>>> >
>>> > But it does seem like people seem to prefer the idea of shifting the Go
>>> > implementation to its own repository. I'd still push for us to include
>>> the
>>> > major version number in the import path, and since we'll have fewer major
>>> > releases and more minor releases, users shouldn't have to update their
>>> > import paths as frequently.
>>> >
>>> > --Matt
>>> >
>>> > On Mon, Jul 22, 2024, 8:37 PM Sutou Kouhei <ko...@clear-code.com> wrote:
>>> >
>>> >> Hi,
>>> >>
>>> >> > Kou, is your plan also counting on moving the
>>> >> > specific nightlies there and removing them from the main repo?
>>> >>
>>> >> Yes. I should have mentioned it explicitly.
>>> >>
>>> >> We will remove most Go related CI jobs from apache/arrow. We
>>> >> will keep Go in integration test CI jobs like we do for
>>> >> apache/arrow-rs.
>>> >>
>>> >>
>>> >> Thanks,
>>> >> --
>>> >> kou
>>> >>
>>> >> In <CA...@mail.gmail.com>
>>> >> "Re: [DISCUSS] Split Go release process" on Fri, 19 Jul 2024 17:14:25
>>> >> +0200,
>>> >> Raúl Cumplido <ra...@gmail.com> wrote:
>>> >>
>>> >> > Hi,
>>> >> >
>>> >> > The conversation around more frequent minor releases and version split
>>> >> > per component has been a long one.
>>> >> >
>>> >> > I am in favour of these changes for the Go implementation because we
>>> >> > have several maintainers.
>>> >> >
>>> >> > It might be difficult to release other implementations that do not
>>> >> > have the same amount of maintainers. I am not sure what our plan is if
>>> >> > one of the split implementations has less maintainers and there's a
>>> >> > requirement for a release (i.e. security fix) but that might be
>>> >> > something to consider in the future.
>>> >> >
>>> >> >> I would defer to Raul and Jacob to corroborate this, but because
>>> >> >> changes to the CI configuration and release verification scripts
>>> don't
>>> >> >> affect other implementations, I have been able to maintain that
>>> >> >> infrastructure myself without too much effort and don't have to lean
>>> >> >> on them for anything except reviews.
>>> >> >
>>> >> > I think releasing and maintaining release scripts / verifications per
>>> >> > component is much easier than for the mono repo. We currently have
>>> >> > over 200 nightly CI jobs in the mono repo that are required to pass
>>> >> > before releasing. Moving some of those to its own repo helps
>>> >> > maintainability. Kou, is your plan also counting on moving the
>>> >> > specific nightlies there and removing them from the main repo?
>>> >> >
>>> >> > I would be in favour of doing a new major release (v18) once the repo
>>> >> > and the changes are in-place to update the import path to something
>>> >> > like:
>>> >> > github.com/apache/arrow-go/v18
>>> >> >
>>> >> > This would avoid confusion with previous releases. We can then follow
>>> >> > up with patch/minor/major as required.
>>> >> >
>>> >> > I am also happy to help with the releases and infrastructure if
>>> >> > necessary as I've done with the main Arrow one (I can also help on
>>> >> > nanoarrow, adbc if necessary).
>>> >> >
>>> >> > Kind regards,
>>> >> > Raul
>>> >> >
>>> >> >
>>> >> >
>>> >> >>
>>> >> >> [1] https://github.com/apache/arrow-nanoarrow/pull/557
>>> >> >>
>>> >> >> On Thu, Jul 18, 2024 at 7:53 PM Matt Topol <zo...@gmail.com>
>>> >> wrote:
>>> >> >> >
>>> >> >> > Part of the goal of splitting out the release processes is that
>>> we'd
>>> >> be
>>> >> >> > able to do minor version releases more frequently instead of major
>>> >> version
>>> >> >> > releases.
>>> >> >> >
>>> >> >> > The general convention in the Go community is to include a major
>>> >> version
>>> >> >> > "v#" in the import path for all major versions past v1 so that if
>>> >> there's a
>>> >> >> > breaking change, it's explicit and prevents potential issues from
>>> >> different
>>> >> >> > major versions being used simultaneously. Being able to do minor
>>> >> version
>>> >> >> > releases more frequently would lead to not having to change the
>>> import
>>> >> >> > paths every 3-6 months, but only if we actually do a breaking
>>> change.
>>> >> >> >
>>> >> >> > On Thu, Jul 18, 2024, 3:55 PM George Godik <gg...@gmail.com>
>>> wrote:
>>> >> >> >
>>> >> >> > > > If we shift the Go lib to a new/different import
>>> >> >> > > path we'll end up with the same problem where people will rely on
>>> >> older
>>> >> >> > > versions and an incorrect path.
>>> >> >> > >
>>> >> >> > > Major version upgrades already require changing the import paths
>>> by
>>> >> >> > > increasing the version. The proposed change would require
>>> everyone
>>> >> to go
>>> >> >> > > through a similar process one last time.
>>> >> >> > >
>>> >> >> > > > More to the point, there would be the question of whether or
>>> not
>>> >> we
>>> >> >> > > should port over the same major version
>>> >> >> > > number, i.e. `github.com/apache/arrow-go/v17`
>>> <http://github.com/apache/arrow-go/v17>
>>> >> <http://github.com/apache/arrow-go/v17>
>>> >> >> > > <http://github.com/apache/arrow-go/v17>
>>> >> >> > > <http://github.com/apache/arrow-go/v17> or something to that
>>> end?
>>> >> Or
>>> >> >> > > do we restart back at v1 (which I think would be confusing)?
>>> >> >> > >
>>> >> >> > > My vote - for whatever it's worth - would be to do away with the
>>> >> >> > > version-in-path naming convention and relying on the go
>>> >> version/package
>>> >> >> > > system for major upgrades.
>>> >> >> > >
>>> >> >> > > Benefits: I don't have to change import paths every 3-6months
>>> >> >> > >
>>> >> >> > > On Thu, Jul 18, 2024 at 3:34 PM Matt Topol <
>>> zotthewizard@gmail.com>
>>> >> wrote:
>>> >> >> > >
>>> >> >> > > > My thoughts:
>>> >> >> > > >
>>> >> >> > > > > * Go doesn't depend on other components such as C++
>>> >> >> > > > > * Go has some active PMC member (Matt) and committer (Joel)
>>> >> >> > > > > * Could you become a release manager for Go?
>>> >> >> > > >
>>> >> >> > > > I'd happily be the release manager for the Go implementation.
>>> >> >> > > >
>>> >> >> > > > > Here is my idea how to proceed this:
>>> >> >> > > > >
>>> >> >> > > > > 1. Extract go/ in apache/arrow to apache/arrow-go like
>>> >> >> > > > > apache/arrow-rs
>>> >> >> > > > > * Filter go/ related commits from apache/arrow and create
>>> >> >> > > > > apache/arrow-go with them like we did for
>>> apache/arrow-rs
>>> >> >> > > > > * Remove go/ related codes from apache/arrow
>>> >> >> > > > > 2. Prepare integration test CI like apache/arrow-rs does:
>>> >> >> > > > >
>>> >> >> > > >
>>> >> >> > > >
>>> >> >> > >
>>> >>
>>> https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml
>>> >> >> > > > > 3. Prepare release script based on apache/arrow-julia,
>>> >> >> > > > > apache/arrow-adbc and/or
>>> apache/arrow-flight-sql-postgresql
>>> >> >> > > >
>>> >> >> > > > Personally I would prefer that we do not extract it to its own
>>> >> separate
>>> >> >> > > > repository purely because I don't want to change the import
>>> path
>>> >> for
>>> >> >> > > users
>>> >> >> > > > again. We already have this issue from before we introduced the
>>> >> major
>>> >> >> > > > version into the import path and shifted it up to allow for the
>>> >> Parquet
>>> >> >> > > lib
>>> >> >> > > > in the same repository. If you look at [1] you see that there's
>>> >> still
>>> >> >> > > over
>>> >> >> > > > 100 projects that never upgraded to v6 or higher because they
>>> are
>>> >> still
>>> >> >> > > > using the old import path. If we shift the Go lib to a
>>> >> new/different
>>> >> >> > > import
>>> >> >> > > > path we'll end up with the same problem where people will rely
>>> on
>>> >> older
>>> >> >> > > > versions and an incorrect path.
>>> >> >> > > >
>>> >> >> > > > If we as a community decide that splitting out the
>>> >> implementations all
>>> >> >> > > into
>>> >> >> > > > separate repositories is the best way forward, I won't hold it
>>> up
>>> >> by
>>> >> >> > > > strictly hammering on this. I'm just concerned about the
>>> >> realities and
>>> >> >> > > > difficulties of communicating the import path change, ensuring
>>> we
>>> >> don't
>>> >> >> > > > break any consumers, and ensuring that users still end up being
>>> >> able to
>>> >> >> > > > upgrade easily.
>>> >> >> > > >
>>> >> >> > > > > The import path could be "github.com/apache/arrow-go"
>>> instead
>>> >> of "
>>> >> >> > > > github.com/apache/arrow-go/arrow". Since go will allow users
>>> to
>>> >> use
>>> >> >> > > > `arrow.Abc` directly if user imports `
>>> github.com/apache/arrow-go` <http://github.com/apache/arrow-go>
>>> >> <http://github.com/apache/arrow-go>
>>> >> >> > > <http://github.com/apache/arrow-go>
>>> >> >> > > > <http://github.com/apache/arrow-go>
>>> >> >> > > > <http://github.com/apache/arrow-go>.
>>> >> >> > > >
>>> >> >> > > > The import path would still have to be `
>>> >> >> > > github.com/apache/arrow-go/arrow`
>>> <http://github.com/apache/arrow-go/arrow>
>>> >> <http://github.com/apache/arrow-go/arrow>
>>> >> >> > > <http://github.com/apache/arrow-go/arrow>
>>> >> >> > > > <http://github.com/apache/arrow-go/arrow>
>>> >> >> > > > since it would also contain the parquet implementation in `
>>> >> >> > > > github.com/apache/arrow-go/parquet`
>>> <http://github.com/apache/arrow-go/parquet>
>>> >> <http://github.com/apache/arrow-go/parquet>
>>> >> >> > > <http://github.com/apache/arrow-go/parquet>
>>> >> >> > > > <http://github.com/apache/arrow-go/parquet>. More to the
>>> point,
>>> >> there
>>> >> >> > > > would be the
>>> >> >> > > > question of whether or not we should port over the same major
>>> >> version
>>> >> >> > > > number, i.e. `github.com/apache/arrow-go/v17`
>>> <http://github.com/apache/arrow-go/v17>
>>> >> <http://github.com/apache/arrow-go/v17>
>>> >> >> > > <http://github.com/apache/arrow-go/v17>
>>> >> >> > > > <http://github.com/apache/arrow-go/v17> or something to that
>>> >> end? Or
>>> >> >> > > > do we restart back at v1 (which I think would be confusing)?
>>> >> >> > > >
>>> >> >> > > > --Matt
>>> >> >> > > >
>>> >> >> > > > [1]: https://pkg.go.dev/github.com/apache/arrow/go/arrow
>>> >> >> > > >
>>> >> >> > > > On Thu, Jul 18, 2024 at 7:33 AM Antoine Pitrou <
>>> >> antoine@python.org>
>>> >> >> > > wrote:
>>> >> >> > > >
>>> >> >> > > > >
>>> >> >> > > > > Hi Kou,
>>> >> >> > > > >
>>> >> >> > > > > Le 18/07/2024 à 11:33, Sutou Kouhei a écrit :
>>> >> >> > > > > >
>>> >> >> > > > > > Here is my idea how to proceed this:
>>> >> >> > > > > >
>>> >> >> > > > > > 1. Extract go/ in apache/arrow to apache/arrow-go like
>>> >> >> > > > > > apache/arrow-rs
>>> >> >> > > > > > * Filter go/ related commits from apache/arrow and
>>> create
>>> >> >> > > > > > apache/arrow-go with them like we did for
>>> >> apache/arrow-rs
>>> >> >> > > > > > * Remove go/ related codes from apache/arrow
>>> >> >> > > > > > 2. Prepare integration test CI like apache/arrow-rs does:
>>> >> >> > > > > >
>>> >> >> > > > >
>>> >> >> > > >
>>> >> >> > >
>>> >>
>>> https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml
>>> >> >> > > > > > 3. Prepare release script based on apache/arrow-julia,
>>> >> >> > > > > > apache/arrow-adbc and/or
>>> >> apache/arrow-flight-sql-postgresql
>>> >> >> > > > >
>>> >> >> > > > > I think this is a good idea, but I'm not part of the Go
>>> >> maintainers.
>>> >> >> > > > >
>>> >> >> > > > > > Cons of this idea:
>>> >> >> > > > > >
>>> >> >> > > > > > * This is a backward incompatible change
>>> >> >> > > > > > * Users need to change their "import" to
>>> >> >> > > > > > "github.com/apache/arrow-go/arrow" from
>>> >> >> > > > > > "github.com/apache/arrow/go/arrow"
>>> >> >> > > > >
>>> >> >> > > > > Is there no way to leave some kind of alias or redirection in
>>> >> the
>>> >> >> > > > > apache/arrow repository?
>>> >> >> > > > >
>>> >> >> > > > > Regards
>>> >> >> > > > >
>>> >> >> > > > > Antoine.
>>> >> >> > > > >
>>> >> >> > > >
>>> >> >> > >
>>> >>
>>>
Re: [DISCUSS] Split Go release process
Posted by Sutou Kouhei <ko...@clear-code.com>.
Hi,
> I think we'll need a vote on it *eventually*, but we probably can wait
> until the repo is up and ready with the vote being when we "pull the
> trigger" so-to-say and turn off Go releases from the main
> github.com/apache/arrow repository and start releasing through
> github.com/apache/arrow-go. We'll also probably need to shift open PRs /
> issues to the new repo after we do this.
OK. I'll start a vote next week if nobody objects this
proposal in this week. I think that a vote after preparing
github.com/apache/arrow-go may inhibit -1 vote. So a vote
before preparing github.com/apache/arrow-go will be fair.
Thanks,
--
kou
In <CA...@mail.gmail.com>
"Re: [DISCUSS] Split Go release process" on Mon, 19 Aug 2024 16:15:52 -0400,
Matt Topol <zo...@gmail.com> wrote:
>> Based on 6., users need to change their import paths on
>> upgrade whether we keep using apache/arrow or we use new
>> apache/arrow-go.
>>
>> If we use new apache/arrow-go, we will be able to reduce
>> maintenance cost for apache/arrow (e.g. we can remove Go
>> related scripts, CI jobs and so on from apache/arrow). Let's
>> use apache/arrow-go.
>>
>> If nobody objects splitting Apache Arrow Go to
>> apache/arrow-go in this week, I'll start working on this
>> next week. (Do we need a vote for this?)
>
> I think we'll need a vote on it *eventually*, but we probably can wait
> until the repo is up and ready with the vote being when we "pull the
> trigger" so-to-say and turn off Go releases from the main
> github.com/apache/arrow repository and start releasing through
> github.com/apache/arrow-go. We'll also probably need to shift open PRs /
> issues to the new repo after we do this.
>
> I'm happy to review any of the scripts being added to the new repository
> and help out if you need it in getting this going. I think the prospect of
> fewer major releases, and thus easier upgrades, is very worthwhile for us
> to pursue this.
>
> --Matt
>
> On Mon, Aug 19, 2024 at 1:14 AM Sutou Kouhei <ko...@clear-code.com> wrote:
>
>> Hi,
>>
>> Sorry for not working on this.
>>
>> Thanks for sharing the standard docs! I've read it and
>> related docs.
>>
>> Here is the summary I learned in this thread and the
>> standard docs:
>>
>> 1. We're using "github.com/apache/arrow/go/v${VERSION}
>> <http://github.com/apache/arrow/go/v$%7BVERSION%7D>" such
>> as "github.com/apache/arrow/go/v17" as our module name
>> * https://pkg.go.dev/github.com/apache/arrow/go/v17/arrow
>> * Including the version number part ("v${VERSION}") is
>> important
>> * Users can avoid unexpected backward incompatibility by
>> this style
>> 2. We used to use "github.com/apache/arrow/go" as our module
>> name in v5 or earlier
>> * https://pkg.go.dev/github.com/apache/arrow/go/arrow
>> * 133 modules still use this
>> 3. We want to avoid user side changes as much as possible
>> * As 2. shows, users may keep using old version if there
>> is any change is required
>> 4. The current users need to change Apache Arrow Go's import
>> path to "github.com/apache/arrow/go/v${VERSION
>> <http://github.com/apache/arrow/go/v$%7BVERSION> + 1}" when
>> they want to upgrade Apache Arrow Go
>> * We don't want to require more changes than "changing
>> import path" for users as mentioned in 3.
>> 5. We can't provide backward compatible module name such as
>> "github.com/apache/arrow/go/v18" for
>> "github.com/apache/arrow-go/v18"
>> * Go doesn't provide the feature
>> 6. We want to keep "v${VERSION}" in our module name even if
>> we split Apache Arrow Go to apache/arrow-go
>> * It's for avoiding unexpected backward incompatibility
>> in users' projects
>>
>>
>> Based on 6., users need to change their import paths on
>> upgrade whether we keep using apache/arrow or we use new
>> apache/arrow-go.
>>
>> If we use new apache/arrow-go, we will be able to reduce
>> maintenance cost for apache/arrow (e.g. we can remove Go
>> related scripts, CI jobs and so on from apache/arrow). Let's
>> use apache/arrow-go.
>>
>> If nobody objects splitting Apache Arrow Go to
>> apache/arrow-go in this week, I'll start working on this
>> next week. (Do we need a vote for this?)
>>
>>
>> Thanks,
>> --
>> kou
>>
>> In <CA...@mail.gmail.com>
>> "Re: [DISCUSS] Split Go release process" on Mon, 22 Jul 2024 20:47:57
>> -0400,
>> Matt Topol <zo...@gmail.com> wrote:
>>
>> > Hey Kou,
>> >
>> > https://go.dev/doc/modules/release-workflow is the standard docs for
>> > developing module versioning and publishing with Go.
>> >
>> > There isn't really a way to alias an import path to a different git repo
>> > because it uses the GitHub URL itself as the import path.
>> >
>> > But it does seem like people seem to prefer the idea of shifting the Go
>> > implementation to its own repository. I'd still push for us to include
>> the
>> > major version number in the import path, and since we'll have fewer major
>> > releases and more minor releases, users shouldn't have to update their
>> > import paths as frequently.
>> >
>> > --Matt
>> >
>> > On Mon, Jul 22, 2024, 8:37 PM Sutou Kouhei <ko...@clear-code.com> wrote:
>> >
>> >> Hi,
>> >>
>> >> > Kou, is your plan also counting on moving the
>> >> > specific nightlies there and removing them from the main repo?
>> >>
>> >> Yes. I should have mentioned it explicitly.
>> >>
>> >> We will remove most Go related CI jobs from apache/arrow. We
>> >> will keep Go in integration test CI jobs like we do for
>> >> apache/arrow-rs.
>> >>
>> >>
>> >> Thanks,
>> >> --
>> >> kou
>> >>
>> >> In <CA...@mail.gmail.com>
>> >> "Re: [DISCUSS] Split Go release process" on Fri, 19 Jul 2024 17:14:25
>> >> +0200,
>> >> Raúl Cumplido <ra...@gmail.com> wrote:
>> >>
>> >> > Hi,
>> >> >
>> >> > The conversation around more frequent minor releases and version split
>> >> > per component has been a long one.
>> >> >
>> >> > I am in favour of these changes for the Go implementation because we
>> >> > have several maintainers.
>> >> >
>> >> > It might be difficult to release other implementations that do not
>> >> > have the same amount of maintainers. I am not sure what our plan is if
>> >> > one of the split implementations has less maintainers and there's a
>> >> > requirement for a release (i.e. security fix) but that might be
>> >> > something to consider in the future.
>> >> >
>> >> >> I would defer to Raul and Jacob to corroborate this, but because
>> >> >> changes to the CI configuration and release verification scripts
>> don't
>> >> >> affect other implementations, I have been able to maintain that
>> >> >> infrastructure myself without too much effort and don't have to lean
>> >> >> on them for anything except reviews.
>> >> >
>> >> > I think releasing and maintaining release scripts / verifications per
>> >> > component is much easier than for the mono repo. We currently have
>> >> > over 200 nightly CI jobs in the mono repo that are required to pass
>> >> > before releasing. Moving some of those to its own repo helps
>> >> > maintainability. Kou, is your plan also counting on moving the
>> >> > specific nightlies there and removing them from the main repo?
>> >> >
>> >> > I would be in favour of doing a new major release (v18) once the repo
>> >> > and the changes are in-place to update the import path to something
>> >> > like:
>> >> > github.com/apache/arrow-go/v18
>> >> >
>> >> > This would avoid confusion with previous releases. We can then follow
>> >> > up with patch/minor/major as required.
>> >> >
>> >> > I am also happy to help with the releases and infrastructure if
>> >> > necessary as I've done with the main Arrow one (I can also help on
>> >> > nanoarrow, adbc if necessary).
>> >> >
>> >> > Kind regards,
>> >> > Raul
>> >> >
>> >> >
>> >> >
>> >> >>
>> >> >> [1] https://github.com/apache/arrow-nanoarrow/pull/557
>> >> >>
>> >> >> On Thu, Jul 18, 2024 at 7:53 PM Matt Topol <zo...@gmail.com>
>> >> wrote:
>> >> >> >
>> >> >> > Part of the goal of splitting out the release processes is that
>> we'd
>> >> be
>> >> >> > able to do minor version releases more frequently instead of major
>> >> version
>> >> >> > releases.
>> >> >> >
>> >> >> > The general convention in the Go community is to include a major
>> >> version
>> >> >> > "v#" in the import path for all major versions past v1 so that if
>> >> there's a
>> >> >> > breaking change, it's explicit and prevents potential issues from
>> >> different
>> >> >> > major versions being used simultaneously. Being able to do minor
>> >> version
>> >> >> > releases more frequently would lead to not having to change the
>> import
>> >> >> > paths every 3-6 months, but only if we actually do a breaking
>> change.
>> >> >> >
>> >> >> > On Thu, Jul 18, 2024, 3:55 PM George Godik <gg...@gmail.com>
>> wrote:
>> >> >> >
>> >> >> > > > If we shift the Go lib to a new/different import
>> >> >> > > path we'll end up with the same problem where people will rely on
>> >> older
>> >> >> > > versions and an incorrect path.
>> >> >> > >
>> >> >> > > Major version upgrades already require changing the import paths
>> by
>> >> >> > > increasing the version. The proposed change would require
>> everyone
>> >> to go
>> >> >> > > through a similar process one last time.
>> >> >> > >
>> >> >> > > > More to the point, there would be the question of whether or
>> not
>> >> we
>> >> >> > > should port over the same major version
>> >> >> > > number, i.e. `github.com/apache/arrow-go/v17`
>> <http://github.com/apache/arrow-go/v17>
>> >> <http://github.com/apache/arrow-go/v17>
>> >> >> > > <http://github.com/apache/arrow-go/v17>
>> >> >> > > <http://github.com/apache/arrow-go/v17> or something to that
>> end?
>> >> Or
>> >> >> > > do we restart back at v1 (which I think would be confusing)?
>> >> >> > >
>> >> >> > > My vote - for whatever it's worth - would be to do away with the
>> >> >> > > version-in-path naming convention and relying on the go
>> >> version/package
>> >> >> > > system for major upgrades.
>> >> >> > >
>> >> >> > > Benefits: I don't have to change import paths every 3-6months
>> >> >> > >
>> >> >> > > On Thu, Jul 18, 2024 at 3:34 PM Matt Topol <
>> zotthewizard@gmail.com>
>> >> wrote:
>> >> >> > >
>> >> >> > > > My thoughts:
>> >> >> > > >
>> >> >> > > > > * Go doesn't depend on other components such as C++
>> >> >> > > > > * Go has some active PMC member (Matt) and committer (Joel)
>> >> >> > > > > * Could you become a release manager for Go?
>> >> >> > > >
>> >> >> > > > I'd happily be the release manager for the Go implementation.
>> >> >> > > >
>> >> >> > > > > Here is my idea how to proceed this:
>> >> >> > > > >
>> >> >> > > > > 1. Extract go/ in apache/arrow to apache/arrow-go like
>> >> >> > > > > apache/arrow-rs
>> >> >> > > > > * Filter go/ related commits from apache/arrow and create
>> >> >> > > > > apache/arrow-go with them like we did for
>> apache/arrow-rs
>> >> >> > > > > * Remove go/ related codes from apache/arrow
>> >> >> > > > > 2. Prepare integration test CI like apache/arrow-rs does:
>> >> >> > > > >
>> >> >> > > >
>> >> >> > > >
>> >> >> > >
>> >>
>> https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml
>> >> >> > > > > 3. Prepare release script based on apache/arrow-julia,
>> >> >> > > > > apache/arrow-adbc and/or
>> apache/arrow-flight-sql-postgresql
>> >> >> > > >
>> >> >> > > > Personally I would prefer that we do not extract it to its own
>> >> separate
>> >> >> > > > repository purely because I don't want to change the import
>> path
>> >> for
>> >> >> > > users
>> >> >> > > > again. We already have this issue from before we introduced the
>> >> major
>> >> >> > > > version into the import path and shifted it up to allow for the
>> >> Parquet
>> >> >> > > lib
>> >> >> > > > in the same repository. If you look at [1] you see that there's
>> >> still
>> >> >> > > over
>> >> >> > > > 100 projects that never upgraded to v6 or higher because they
>> are
>> >> still
>> >> >> > > > using the old import path. If we shift the Go lib to a
>> >> new/different
>> >> >> > > import
>> >> >> > > > path we'll end up with the same problem where people will rely
>> on
>> >> older
>> >> >> > > > versions and an incorrect path.
>> >> >> > > >
>> >> >> > > > If we as a community decide that splitting out the
>> >> implementations all
>> >> >> > > into
>> >> >> > > > separate repositories is the best way forward, I won't hold it
>> up
>> >> by
>> >> >> > > > strictly hammering on this. I'm just concerned about the
>> >> realities and
>> >> >> > > > difficulties of communicating the import path change, ensuring
>> we
>> >> don't
>> >> >> > > > break any consumers, and ensuring that users still end up being
>> >> able to
>> >> >> > > > upgrade easily.
>> >> >> > > >
>> >> >> > > > > The import path could be "github.com/apache/arrow-go"
>> instead
>> >> of "
>> >> >> > > > github.com/apache/arrow-go/arrow". Since go will allow users
>> to
>> >> use
>> >> >> > > > `arrow.Abc` directly if user imports `
>> github.com/apache/arrow-go` <http://github.com/apache/arrow-go>
>> >> <http://github.com/apache/arrow-go>
>> >> >> > > <http://github.com/apache/arrow-go>
>> >> >> > > > <http://github.com/apache/arrow-go>
>> >> >> > > > <http://github.com/apache/arrow-go>.
>> >> >> > > >
>> >> >> > > > The import path would still have to be `
>> >> >> > > github.com/apache/arrow-go/arrow`
>> <http://github.com/apache/arrow-go/arrow>
>> >> <http://github.com/apache/arrow-go/arrow>
>> >> >> > > <http://github.com/apache/arrow-go/arrow>
>> >> >> > > > <http://github.com/apache/arrow-go/arrow>
>> >> >> > > > since it would also contain the parquet implementation in `
>> >> >> > > > github.com/apache/arrow-go/parquet`
>> <http://github.com/apache/arrow-go/parquet>
>> >> <http://github.com/apache/arrow-go/parquet>
>> >> >> > > <http://github.com/apache/arrow-go/parquet>
>> >> >> > > > <http://github.com/apache/arrow-go/parquet>. More to the
>> point,
>> >> there
>> >> >> > > > would be the
>> >> >> > > > question of whether or not we should port over the same major
>> >> version
>> >> >> > > > number, i.e. `github.com/apache/arrow-go/v17`
>> <http://github.com/apache/arrow-go/v17>
>> >> <http://github.com/apache/arrow-go/v17>
>> >> >> > > <http://github.com/apache/arrow-go/v17>
>> >> >> > > > <http://github.com/apache/arrow-go/v17> or something to that
>> >> end? Or
>> >> >> > > > do we restart back at v1 (which I think would be confusing)?
>> >> >> > > >
>> >> >> > > > --Matt
>> >> >> > > >
>> >> >> > > > [1]: https://pkg.go.dev/github.com/apache/arrow/go/arrow
>> >> >> > > >
>> >> >> > > > On Thu, Jul 18, 2024 at 7:33 AM Antoine Pitrou <
>> >> antoine@python.org>
>> >> >> > > wrote:
>> >> >> > > >
>> >> >> > > > >
>> >> >> > > > > Hi Kou,
>> >> >> > > > >
>> >> >> > > > > Le 18/07/2024 à 11:33, Sutou Kouhei a écrit :
>> >> >> > > > > >
>> >> >> > > > > > Here is my idea how to proceed this:
>> >> >> > > > > >
>> >> >> > > > > > 1. Extract go/ in apache/arrow to apache/arrow-go like
>> >> >> > > > > > apache/arrow-rs
>> >> >> > > > > > * Filter go/ related commits from apache/arrow and
>> create
>> >> >> > > > > > apache/arrow-go with them like we did for
>> >> apache/arrow-rs
>> >> >> > > > > > * Remove go/ related codes from apache/arrow
>> >> >> > > > > > 2. Prepare integration test CI like apache/arrow-rs does:
>> >> >> > > > > >
>> >> >> > > > >
>> >> >> > > >
>> >> >> > >
>> >>
>> https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml
>> >> >> > > > > > 3. Prepare release script based on apache/arrow-julia,
>> >> >> > > > > > apache/arrow-adbc and/or
>> >> apache/arrow-flight-sql-postgresql
>> >> >> > > > >
>> >> >> > > > > I think this is a good idea, but I'm not part of the Go
>> >> maintainers.
>> >> >> > > > >
>> >> >> > > > > > Cons of this idea:
>> >> >> > > > > >
>> >> >> > > > > > * This is a backward incompatible change
>> >> >> > > > > > * Users need to change their "import" to
>> >> >> > > > > > "github.com/apache/arrow-go/arrow" from
>> >> >> > > > > > "github.com/apache/arrow/go/arrow"
>> >> >> > > > >
>> >> >> > > > > Is there no way to leave some kind of alias or redirection in
>> >> the
>> >> >> > > > > apache/arrow repository?
>> >> >> > > > >
>> >> >> > > > > Regards
>> >> >> > > > >
>> >> >> > > > > Antoine.
>> >> >> > > > >
>> >> >> > > >
>> >> >> > >
>> >>
>>
Re: [DISCUSS] Split Go release process
Posted by Joel Lubinitsky <jo...@gmail.com>.
This all looks good to me, and I'm happy to review/help with any parts of
the migration as well.
Thanks,
Joel
On Mon, Aug 19, 2024 at 4:16 PM Matt Topol <zo...@gmail.com> wrote:
> > Based on 6., users need to change their import paths on
> > upgrade whether we keep using apache/arrow or we use new
> > apache/arrow-go.
> >
> > If we use new apache/arrow-go, we will be able to reduce
> > maintenance cost for apache/arrow (e.g. we can remove Go
> > related scripts, CI jobs and so on from apache/arrow). Let's
> > use apache/arrow-go.
> >
> > If nobody objects splitting Apache Arrow Go to
> > apache/arrow-go in this week, I'll start working on this
> > next week. (Do we need a vote for this?)
>
> I think we'll need a vote on it *eventually*, but we probably can wait
> until the repo is up and ready with the vote being when we "pull the
> trigger" so-to-say and turn off Go releases from the main
> github.com/apache/arrow repository and start releasing through
> github.com/apache/arrow-go. We'll also probably need to shift open PRs /
> issues to the new repo after we do this.
>
> I'm happy to review any of the scripts being added to the new repository
> and help out if you need it in getting this going. I think the prospect of
> fewer major releases, and thus easier upgrades, is very worthwhile for us
> to pursue this.
>
> --Matt
>
> On Mon, Aug 19, 2024 at 1:14 AM Sutou Kouhei <ko...@clear-code.com> wrote:
>
> > Hi,
> >
> > Sorry for not working on this.
> >
> > Thanks for sharing the standard docs! I've read it and
> > related docs.
> >
> > Here is the summary I learned in this thread and the
> > standard docs:
> >
> > 1. We're using "github.com/apache/arrow/go/v${VERSION}
> <http://github.com/apache/arrow/go/v$%7BVERSION%7D>
> > <http://github.com/apache/arrow/go/v$%7BVERSION%7D>" such
> > as "github.com/apache/arrow/go/v17" as our module name
> > * https://pkg.go.dev/github.com/apache/arrow/go/v17/arrow
> > * Including the version number part ("v${VERSION}") is
> > important
> > * Users can avoid unexpected backward incompatibility by
> > this style
> > 2. We used to use "github.com/apache/arrow/go" as our module
> > name in v5 or earlier
> > * https://pkg.go.dev/github.com/apache/arrow/go/arrow
> > * 133 modules still use this
> > 3. We want to avoid user side changes as much as possible
> > * As 2. shows, users may keep using old version if there
> > is any change is required
> > 4. The current users need to change Apache Arrow Go's import
> > path to "github.com/apache/arrow/go/v${VERSION
> <http://github.com/apache/arrow/go/v$%7BVERSION>
> > <http://github.com/apache/arrow/go/v$%7BVERSION> + 1}" when
> > they want to upgrade Apache Arrow Go
> > * We don't want to require more changes than "changing
> > import path" for users as mentioned in 3.
> > 5. We can't provide backward compatible module name such as
> > "github.com/apache/arrow/go/v18" for
> > "github.com/apache/arrow-go/v18"
> > * Go doesn't provide the feature
> > 6. We want to keep "v${VERSION}" in our module name even if
> > we split Apache Arrow Go to apache/arrow-go
> > * It's for avoiding unexpected backward incompatibility
> > in users' projects
> >
> >
> > Based on 6., users need to change their import paths on
> > upgrade whether we keep using apache/arrow or we use new
> > apache/arrow-go.
> >
> > If we use new apache/arrow-go, we will be able to reduce
> > maintenance cost for apache/arrow (e.g. we can remove Go
> > related scripts, CI jobs and so on from apache/arrow). Let's
> > use apache/arrow-go.
> >
> > If nobody objects splitting Apache Arrow Go to
> > apache/arrow-go in this week, I'll start working on this
> > next week. (Do we need a vote for this?)
> >
> >
> > Thanks,
> > --
> > kou
> >
> > In <CA...@mail.gmail.com>
> > "Re: [DISCUSS] Split Go release process" on Mon, 22 Jul 2024 20:47:57
> > -0400,
> > Matt Topol <zo...@gmail.com> wrote:
> >
> > > Hey Kou,
> > >
> > > https://go.dev/doc/modules/release-workflow is the standard docs for
> > > developing module versioning and publishing with Go.
> > >
> > > There isn't really a way to alias an import path to a different git
> repo
> > > because it uses the GitHub URL itself as the import path.
> > >
> > > But it does seem like people seem to prefer the idea of shifting the Go
> > > implementation to its own repository. I'd still push for us to include
> > the
> > > major version number in the import path, and since we'll have fewer
> major
> > > releases and more minor releases, users shouldn't have to update their
> > > import paths as frequently.
> > >
> > > --Matt
> > >
> > > On Mon, Jul 22, 2024, 8:37 PM Sutou Kouhei <ko...@clear-code.com> wrote:
> > >
> > >> Hi,
> > >>
> > >> > Kou, is your plan also counting on moving the
> > >> > specific nightlies there and removing them from the main repo?
> > >>
> > >> Yes. I should have mentioned it explicitly.
> > >>
> > >> We will remove most Go related CI jobs from apache/arrow. We
> > >> will keep Go in integration test CI jobs like we do for
> > >> apache/arrow-rs.
> > >>
> > >>
> > >> Thanks,
> > >> --
> > >> kou
> > >>
> > >> In <
> CAD1Rbrr2vtXAuNppfrRjGFd+OfCa3q4F+yR6NpKu4TTZLX22XA@mail.gmail.com>
> > >> "Re: [DISCUSS] Split Go release process" on Fri, 19 Jul 2024
> 17:14:25
> > >> +0200,
> > >> Raúl Cumplido <ra...@gmail.com> wrote:
> > >>
> > >> > Hi,
> > >> >
> > >> > The conversation around more frequent minor releases and version
> split
> > >> > per component has been a long one.
> > >> >
> > >> > I am in favour of these changes for the Go implementation because we
> > >> > have several maintainers.
> > >> >
> > >> > It might be difficult to release other implementations that do not
> > >> > have the same amount of maintainers. I am not sure what our plan is
> if
> > >> > one of the split implementations has less maintainers and there's a
> > >> > requirement for a release (i.e. security fix) but that might be
> > >> > something to consider in the future.
> > >> >
> > >> >> I would defer to Raul and Jacob to corroborate this, but because
> > >> >> changes to the CI configuration and release verification scripts
> > don't
> > >> >> affect other implementations, I have been able to maintain that
> > >> >> infrastructure myself without too much effort and don't have to
> lean
> > >> >> on them for anything except reviews.
> > >> >
> > >> > I think releasing and maintaining release scripts / verifications
> per
> > >> > component is much easier than for the mono repo. We currently have
> > >> > over 200 nightly CI jobs in the mono repo that are required to pass
> > >> > before releasing. Moving some of those to its own repo helps
> > >> > maintainability. Kou, is your plan also counting on moving the
> > >> > specific nightlies there and removing them from the main repo?
> > >> >
> > >> > I would be in favour of doing a new major release (v18) once the
> repo
> > >> > and the changes are in-place to update the import path to something
> > >> > like:
> > >> > github.com/apache/arrow-go/v18
> > >> >
> > >> > This would avoid confusion with previous releases. We can then
> follow
> > >> > up with patch/minor/major as required.
> > >> >
> > >> > I am also happy to help with the releases and infrastructure if
> > >> > necessary as I've done with the main Arrow one (I can also help on
> > >> > nanoarrow, adbc if necessary).
> > >> >
> > >> > Kind regards,
> > >> > Raul
> > >> >
> > >> >
> > >> >
> > >> >>
> > >> >> [1] https://github.com/apache/arrow-nanoarrow/pull/557
> > >> >>
> > >> >> On Thu, Jul 18, 2024 at 7:53 PM Matt Topol <zotthewizard@gmail.com
> >
> > >> wrote:
> > >> >> >
> > >> >> > Part of the goal of splitting out the release processes is that
> > we'd
> > >> be
> > >> >> > able to do minor version releases more frequently instead of
> major
> > >> version
> > >> >> > releases.
> > >> >> >
> > >> >> > The general convention in the Go community is to include a major
> > >> version
> > >> >> > "v#" in the import path for all major versions past v1 so that if
> > >> there's a
> > >> >> > breaking change, it's explicit and prevents potential issues from
> > >> different
> > >> >> > major versions being used simultaneously. Being able to do minor
> > >> version
> > >> >> > releases more frequently would lead to not having to change the
> > import
> > >> >> > paths every 3-6 months, but only if we actually do a breaking
> > change.
> > >> >> >
> > >> >> > On Thu, Jul 18, 2024, 3:55 PM George Godik <gg...@gmail.com>
> > wrote:
> > >> >> >
> > >> >> > > > If we shift the Go lib to a new/different import
> > >> >> > > path we'll end up with the same problem where people will rely
> on
> > >> older
> > >> >> > > versions and an incorrect path.
> > >> >> > >
> > >> >> > > Major version upgrades already require changing the import
> paths
> > by
> > >> >> > > increasing the version. The proposed change would require
> > everyone
> > >> to go
> > >> >> > > through a similar process one last time.
> > >> >> > >
> > >> >> > > > More to the point, there would be the question of whether or
> > not
> > >> we
> > >> >> > > should port over the same major version
> > >> >> > > number, i.e. `github.com/apache/arrow-go/v17`
> <http://github.com/apache/arrow-go/v17>
> > <http://github.com/apache/arrow-go/v17>
> > >> <http://github.com/apache/arrow-go/v17>
> > >> >> > > <http://github.com/apache/arrow-go/v17>
> > >> >> > > <http://github.com/apache/arrow-go/v17> or something to that
> > end?
> > >> Or
> > >> >> > > do we restart back at v1 (which I think would be confusing)?
> > >> >> > >
> > >> >> > > My vote - for whatever it's worth - would be to do away with
> the
> > >> >> > > version-in-path naming convention and relying on the go
> > >> version/package
> > >> >> > > system for major upgrades.
> > >> >> > >
> > >> >> > > Benefits: I don't have to change import paths every 3-6months
> > >> >> > >
> > >> >> > > On Thu, Jul 18, 2024 at 3:34 PM Matt Topol <
> > zotthewizard@gmail.com>
> > >> wrote:
> > >> >> > >
> > >> >> > > > My thoughts:
> > >> >> > > >
> > >> >> > > > > * Go doesn't depend on other components such as C++
> > >> >> > > > > * Go has some active PMC member (Matt) and committer (Joel)
> > >> >> > > > > * Could you become a release manager for Go?
> > >> >> > > >
> > >> >> > > > I'd happily be the release manager for the Go implementation.
> > >> >> > > >
> > >> >> > > > > Here is my idea how to proceed this:
> > >> >> > > > >
> > >> >> > > > > 1. Extract go/ in apache/arrow to apache/arrow-go like
> > >> >> > > > > apache/arrow-rs
> > >> >> > > > > * Filter go/ related commits from apache/arrow and
> create
> > >> >> > > > > apache/arrow-go with them like we did for
> > apache/arrow-rs
> > >> >> > > > > * Remove go/ related codes from apache/arrow
> > >> >> > > > > 2. Prepare integration test CI like apache/arrow-rs does:
> > >> >> > > > >
> > >> >> > > >
> > >> >> > > >
> > >> >> > >
> > >>
> >
> https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml
> > >> >> > > > > 3. Prepare release script based on apache/arrow-julia,
> > >> >> > > > > apache/arrow-adbc and/or
> > apache/arrow-flight-sql-postgresql
> > >> >> > > >
> > >> >> > > > Personally I would prefer that we do not extract it to its
> own
> > >> separate
> > >> >> > > > repository purely because I don't want to change the import
> > path
> > >> for
> > >> >> > > users
> > >> >> > > > again. We already have this issue from before we introduced
> the
> > >> major
> > >> >> > > > version into the import path and shifted it up to allow for
> the
> > >> Parquet
> > >> >> > > lib
> > >> >> > > > in the same repository. If you look at [1] you see that
> there's
> > >> still
> > >> >> > > over
> > >> >> > > > 100 projects that never upgraded to v6 or higher because they
> > are
> > >> still
> > >> >> > > > using the old import path. If we shift the Go lib to a
> > >> new/different
> > >> >> > > import
> > >> >> > > > path we'll end up with the same problem where people will
> rely
> > on
> > >> older
> > >> >> > > > versions and an incorrect path.
> > >> >> > > >
> > >> >> > > > If we as a community decide that splitting out the
> > >> implementations all
> > >> >> > > into
> > >> >> > > > separate repositories is the best way forward, I won't hold
> it
> > up
> > >> by
> > >> >> > > > strictly hammering on this. I'm just concerned about the
> > >> realities and
> > >> >> > > > difficulties of communicating the import path change,
> ensuring
> > we
> > >> don't
> > >> >> > > > break any consumers, and ensuring that users still end up
> being
> > >> able to
> > >> >> > > > upgrade easily.
> > >> >> > > >
> > >> >> > > > > The import path could be "github.com/apache/arrow-go"
> > instead
> > >> of "
> > >> >> > > > github.com/apache/arrow-go/arrow". Since go will allow users
> > to
> > >> use
> > >> >> > > > `arrow.Abc` directly if user imports `
> > github.com/apache/arrow-go` <http://github.com/apache/arrow-go> <
> http://github.com/apache/arrow-go>
> > >> <http://github.com/apache/arrow-go>
> > >> >> > > <http://github.com/apache/arrow-go>
> > >> >> > > > <http://github.com/apache/arrow-go>
> > >> >> > > > <http://github.com/apache/arrow-go>.
> > >> >> > > >
> > >> >> > > > The import path would still have to be `
> > >> >> > > github.com/apache/arrow-go/arrow`
> <http://github.com/apache/arrow-go/arrow>
> > <http://github.com/apache/arrow-go/arrow>
> > >> <http://github.com/apache/arrow-go/arrow>
> > >> >> > > <http://github.com/apache/arrow-go/arrow>
> > >> >> > > > <http://github.com/apache/arrow-go/arrow>
> > >> >> > > > since it would also contain the parquet implementation in `
> > >> >> > > > github.com/apache/arrow-go/parquet`
> <http://github.com/apache/arrow-go/parquet>
> > <http://github.com/apache/arrow-go/parquet>
> > >> <http://github.com/apache/arrow-go/parquet>
> > >> >> > > <http://github.com/apache/arrow-go/parquet>
> > >> >> > > > <http://github.com/apache/arrow-go/parquet>. More to the
> > point,
> > >> there
> > >> >> > > > would be the
> > >> >> > > > question of whether or not we should port over the same major
> > >> version
> > >> >> > > > number, i.e. `github.com/apache/arrow-go/v17`
> <http://github.com/apache/arrow-go/v17>
> > <http://github.com/apache/arrow-go/v17>
> > >> <http://github.com/apache/arrow-go/v17>
> > >> >> > > <http://github.com/apache/arrow-go/v17>
> > >> >> > > > <http://github.com/apache/arrow-go/v17> or something to that
> > >> end? Or
> > >> >> > > > do we restart back at v1 (which I think would be confusing)?
> > >> >> > > >
> > >> >> > > > --Matt
> > >> >> > > >
> > >> >> > > > [1]: https://pkg.go.dev/github.com/apache/arrow/go/arrow
> > >> >> > > >
> > >> >> > > > On Thu, Jul 18, 2024 at 7:33 AM Antoine Pitrou <
> > >> antoine@python.org>
> > >> >> > > wrote:
> > >> >> > > >
> > >> >> > > > >
> > >> >> > > > > Hi Kou,
> > >> >> > > > >
> > >> >> > > > > Le 18/07/2024 à 11:33, Sutou Kouhei a écrit :
> > >> >> > > > > >
> > >> >> > > > > > Here is my idea how to proceed this:
> > >> >> > > > > >
> > >> >> > > > > > 1. Extract go/ in apache/arrow to apache/arrow-go like
> > >> >> > > > > > apache/arrow-rs
> > >> >> > > > > > * Filter go/ related commits from apache/arrow and
> > create
> > >> >> > > > > > apache/arrow-go with them like we did for
> > >> apache/arrow-rs
> > >> >> > > > > > * Remove go/ related codes from apache/arrow
> > >> >> > > > > > 2. Prepare integration test CI like apache/arrow-rs does:
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >>
> >
> https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml
> > >> >> > > > > > 3. Prepare release script based on apache/arrow-julia,
> > >> >> > > > > > apache/arrow-adbc and/or
> > >> apache/arrow-flight-sql-postgresql
> > >> >> > > > >
> > >> >> > > > > I think this is a good idea, but I'm not part of the Go
> > >> maintainers.
> > >> >> > > > >
> > >> >> > > > > > Cons of this idea:
> > >> >> > > > > >
> > >> >> > > > > > * This is a backward incompatible change
> > >> >> > > > > > * Users need to change their "import" to
> > >> >> > > > > > "github.com/apache/arrow-go/arrow" from
> > >> >> > > > > > "github.com/apache/arrow/go/arrow"
> > >> >> > > > >
> > >> >> > > > > Is there no way to leave some kind of alias or redirection
> in
> > >> the
> > >> >> > > > > apache/arrow repository?
> > >> >> > > > >
> > >> >> > > > > Regards
> > >> >> > > > >
> > >> >> > > > > Antoine.
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >>
> >
>
Re: [DISCUSS] Split Go release process
Posted by Matt Topol <zo...@gmail.com>.
> Based on 6., users need to change their import paths on
> upgrade whether we keep using apache/arrow or we use new
> apache/arrow-go.
>
> If we use new apache/arrow-go, we will be able to reduce
> maintenance cost for apache/arrow (e.g. we can remove Go
> related scripts, CI jobs and so on from apache/arrow). Let's
> use apache/arrow-go.
>
> If nobody objects splitting Apache Arrow Go to
> apache/arrow-go in this week, I'll start working on this
> next week. (Do we need a vote for this?)
I think we'll need a vote on it *eventually*, but we probably can wait
until the repo is up and ready with the vote being when we "pull the
trigger" so-to-say and turn off Go releases from the main
github.com/apache/arrow repository and start releasing through
github.com/apache/arrow-go. We'll also probably need to shift open PRs /
issues to the new repo after we do this.
I'm happy to review any of the scripts being added to the new repository
and help out if you need it in getting this going. I think the prospect of
fewer major releases, and thus easier upgrades, is very worthwhile for us
to pursue this.
--Matt
On Mon, Aug 19, 2024 at 1:14 AM Sutou Kouhei <ko...@clear-code.com> wrote:
> Hi,
>
> Sorry for not working on this.
>
> Thanks for sharing the standard docs! I've read it and
> related docs.
>
> Here is the summary I learned in this thread and the
> standard docs:
>
> 1. We're using "github.com/apache/arrow/go/v${VERSION}
> <http://github.com/apache/arrow/go/v$%7BVERSION%7D>" such
> as "github.com/apache/arrow/go/v17" as our module name
> * https://pkg.go.dev/github.com/apache/arrow/go/v17/arrow
> * Including the version number part ("v${VERSION}") is
> important
> * Users can avoid unexpected backward incompatibility by
> this style
> 2. We used to use "github.com/apache/arrow/go" as our module
> name in v5 or earlier
> * https://pkg.go.dev/github.com/apache/arrow/go/arrow
> * 133 modules still use this
> 3. We want to avoid user side changes as much as possible
> * As 2. shows, users may keep using old version if there
> is any change is required
> 4. The current users need to change Apache Arrow Go's import
> path to "github.com/apache/arrow/go/v${VERSION
> <http://github.com/apache/arrow/go/v$%7BVERSION> + 1}" when
> they want to upgrade Apache Arrow Go
> * We don't want to require more changes than "changing
> import path" for users as mentioned in 3.
> 5. We can't provide backward compatible module name such as
> "github.com/apache/arrow/go/v18" for
> "github.com/apache/arrow-go/v18"
> * Go doesn't provide the feature
> 6. We want to keep "v${VERSION}" in our module name even if
> we split Apache Arrow Go to apache/arrow-go
> * It's for avoiding unexpected backward incompatibility
> in users' projects
>
>
> Based on 6., users need to change their import paths on
> upgrade whether we keep using apache/arrow or we use new
> apache/arrow-go.
>
> If we use new apache/arrow-go, we will be able to reduce
> maintenance cost for apache/arrow (e.g. we can remove Go
> related scripts, CI jobs and so on from apache/arrow). Let's
> use apache/arrow-go.
>
> If nobody objects splitting Apache Arrow Go to
> apache/arrow-go in this week, I'll start working on this
> next week. (Do we need a vote for this?)
>
>
> Thanks,
> --
> kou
>
> In <CA...@mail.gmail.com>
> "Re: [DISCUSS] Split Go release process" on Mon, 22 Jul 2024 20:47:57
> -0400,
> Matt Topol <zo...@gmail.com> wrote:
>
> > Hey Kou,
> >
> > https://go.dev/doc/modules/release-workflow is the standard docs for
> > developing module versioning and publishing with Go.
> >
> > There isn't really a way to alias an import path to a different git repo
> > because it uses the GitHub URL itself as the import path.
> >
> > But it does seem like people seem to prefer the idea of shifting the Go
> > implementation to its own repository. I'd still push for us to include
> the
> > major version number in the import path, and since we'll have fewer major
> > releases and more minor releases, users shouldn't have to update their
> > import paths as frequently.
> >
> > --Matt
> >
> > On Mon, Jul 22, 2024, 8:37 PM Sutou Kouhei <ko...@clear-code.com> wrote:
> >
> >> Hi,
> >>
> >> > Kou, is your plan also counting on moving the
> >> > specific nightlies there and removing them from the main repo?
> >>
> >> Yes. I should have mentioned it explicitly.
> >>
> >> We will remove most Go related CI jobs from apache/arrow. We
> >> will keep Go in integration test CI jobs like we do for
> >> apache/arrow-rs.
> >>
> >>
> >> Thanks,
> >> --
> >> kou
> >>
> >> In <CA...@mail.gmail.com>
> >> "Re: [DISCUSS] Split Go release process" on Fri, 19 Jul 2024 17:14:25
> >> +0200,
> >> Raúl Cumplido <ra...@gmail.com> wrote:
> >>
> >> > Hi,
> >> >
> >> > The conversation around more frequent minor releases and version split
> >> > per component has been a long one.
> >> >
> >> > I am in favour of these changes for the Go implementation because we
> >> > have several maintainers.
> >> >
> >> > It might be difficult to release other implementations that do not
> >> > have the same amount of maintainers. I am not sure what our plan is if
> >> > one of the split implementations has less maintainers and there's a
> >> > requirement for a release (i.e. security fix) but that might be
> >> > something to consider in the future.
> >> >
> >> >> I would defer to Raul and Jacob to corroborate this, but because
> >> >> changes to the CI configuration and release verification scripts
> don't
> >> >> affect other implementations, I have been able to maintain that
> >> >> infrastructure myself without too much effort and don't have to lean
> >> >> on them for anything except reviews.
> >> >
> >> > I think releasing and maintaining release scripts / verifications per
> >> > component is much easier than for the mono repo. We currently have
> >> > over 200 nightly CI jobs in the mono repo that are required to pass
> >> > before releasing. Moving some of those to its own repo helps
> >> > maintainability. Kou, is your plan also counting on moving the
> >> > specific nightlies there and removing them from the main repo?
> >> >
> >> > I would be in favour of doing a new major release (v18) once the repo
> >> > and the changes are in-place to update the import path to something
> >> > like:
> >> > github.com/apache/arrow-go/v18
> >> >
> >> > This would avoid confusion with previous releases. We can then follow
> >> > up with patch/minor/major as required.
> >> >
> >> > I am also happy to help with the releases and infrastructure if
> >> > necessary as I've done with the main Arrow one (I can also help on
> >> > nanoarrow, adbc if necessary).
> >> >
> >> > Kind regards,
> >> > Raul
> >> >
> >> >
> >> >
> >> >>
> >> >> [1] https://github.com/apache/arrow-nanoarrow/pull/557
> >> >>
> >> >> On Thu, Jul 18, 2024 at 7:53 PM Matt Topol <zo...@gmail.com>
> >> wrote:
> >> >> >
> >> >> > Part of the goal of splitting out the release processes is that
> we'd
> >> be
> >> >> > able to do minor version releases more frequently instead of major
> >> version
> >> >> > releases.
> >> >> >
> >> >> > The general convention in the Go community is to include a major
> >> version
> >> >> > "v#" in the import path for all major versions past v1 so that if
> >> there's a
> >> >> > breaking change, it's explicit and prevents potential issues from
> >> different
> >> >> > major versions being used simultaneously. Being able to do minor
> >> version
> >> >> > releases more frequently would lead to not having to change the
> import
> >> >> > paths every 3-6 months, but only if we actually do a breaking
> change.
> >> >> >
> >> >> > On Thu, Jul 18, 2024, 3:55 PM George Godik <gg...@gmail.com>
> wrote:
> >> >> >
> >> >> > > > If we shift the Go lib to a new/different import
> >> >> > > path we'll end up with the same problem where people will rely on
> >> older
> >> >> > > versions and an incorrect path.
> >> >> > >
> >> >> > > Major version upgrades already require changing the import paths
> by
> >> >> > > increasing the version. The proposed change would require
> everyone
> >> to go
> >> >> > > through a similar process one last time.
> >> >> > >
> >> >> > > > More to the point, there would be the question of whether or
> not
> >> we
> >> >> > > should port over the same major version
> >> >> > > number, i.e. `github.com/apache/arrow-go/v17`
> <http://github.com/apache/arrow-go/v17>
> >> <http://github.com/apache/arrow-go/v17>
> >> >> > > <http://github.com/apache/arrow-go/v17>
> >> >> > > <http://github.com/apache/arrow-go/v17> or something to that
> end?
> >> Or
> >> >> > > do we restart back at v1 (which I think would be confusing)?
> >> >> > >
> >> >> > > My vote - for whatever it's worth - would be to do away with the
> >> >> > > version-in-path naming convention and relying on the go
> >> version/package
> >> >> > > system for major upgrades.
> >> >> > >
> >> >> > > Benefits: I don't have to change import paths every 3-6months
> >> >> > >
> >> >> > > On Thu, Jul 18, 2024 at 3:34 PM Matt Topol <
> zotthewizard@gmail.com>
> >> wrote:
> >> >> > >
> >> >> > > > My thoughts:
> >> >> > > >
> >> >> > > > > * Go doesn't depend on other components such as C++
> >> >> > > > > * Go has some active PMC member (Matt) and committer (Joel)
> >> >> > > > > * Could you become a release manager for Go?
> >> >> > > >
> >> >> > > > I'd happily be the release manager for the Go implementation.
> >> >> > > >
> >> >> > > > > Here is my idea how to proceed this:
> >> >> > > > >
> >> >> > > > > 1. Extract go/ in apache/arrow to apache/arrow-go like
> >> >> > > > > apache/arrow-rs
> >> >> > > > > * Filter go/ related commits from apache/arrow and create
> >> >> > > > > apache/arrow-go with them like we did for
> apache/arrow-rs
> >> >> > > > > * Remove go/ related codes from apache/arrow
> >> >> > > > > 2. Prepare integration test CI like apache/arrow-rs does:
> >> >> > > > >
> >> >> > > >
> >> >> > > >
> >> >> > >
> >>
> https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml
> >> >> > > > > 3. Prepare release script based on apache/arrow-julia,
> >> >> > > > > apache/arrow-adbc and/or
> apache/arrow-flight-sql-postgresql
> >> >> > > >
> >> >> > > > Personally I would prefer that we do not extract it to its own
> >> separate
> >> >> > > > repository purely because I don't want to change the import
> path
> >> for
> >> >> > > users
> >> >> > > > again. We already have this issue from before we introduced the
> >> major
> >> >> > > > version into the import path and shifted it up to allow for the
> >> Parquet
> >> >> > > lib
> >> >> > > > in the same repository. If you look at [1] you see that there's
> >> still
> >> >> > > over
> >> >> > > > 100 projects that never upgraded to v6 or higher because they
> are
> >> still
> >> >> > > > using the old import path. If we shift the Go lib to a
> >> new/different
> >> >> > > import
> >> >> > > > path we'll end up with the same problem where people will rely
> on
> >> older
> >> >> > > > versions and an incorrect path.
> >> >> > > >
> >> >> > > > If we as a community decide that splitting out the
> >> implementations all
> >> >> > > into
> >> >> > > > separate repositories is the best way forward, I won't hold it
> up
> >> by
> >> >> > > > strictly hammering on this. I'm just concerned about the
> >> realities and
> >> >> > > > difficulties of communicating the import path change, ensuring
> we
> >> don't
> >> >> > > > break any consumers, and ensuring that users still end up being
> >> able to
> >> >> > > > upgrade easily.
> >> >> > > >
> >> >> > > > > The import path could be "github.com/apache/arrow-go"
> instead
> >> of "
> >> >> > > > github.com/apache/arrow-go/arrow". Since go will allow users
> to
> >> use
> >> >> > > > `arrow.Abc` directly if user imports `
> github.com/apache/arrow-go` <http://github.com/apache/arrow-go>
> >> <http://github.com/apache/arrow-go>
> >> >> > > <http://github.com/apache/arrow-go>
> >> >> > > > <http://github.com/apache/arrow-go>
> >> >> > > > <http://github.com/apache/arrow-go>.
> >> >> > > >
> >> >> > > > The import path would still have to be `
> >> >> > > github.com/apache/arrow-go/arrow`
> <http://github.com/apache/arrow-go/arrow>
> >> <http://github.com/apache/arrow-go/arrow>
> >> >> > > <http://github.com/apache/arrow-go/arrow>
> >> >> > > > <http://github.com/apache/arrow-go/arrow>
> >> >> > > > since it would also contain the parquet implementation in `
> >> >> > > > github.com/apache/arrow-go/parquet`
> <http://github.com/apache/arrow-go/parquet>
> >> <http://github.com/apache/arrow-go/parquet>
> >> >> > > <http://github.com/apache/arrow-go/parquet>
> >> >> > > > <http://github.com/apache/arrow-go/parquet>. More to the
> point,
> >> there
> >> >> > > > would be the
> >> >> > > > question of whether or not we should port over the same major
> >> version
> >> >> > > > number, i.e. `github.com/apache/arrow-go/v17`
> <http://github.com/apache/arrow-go/v17>
> >> <http://github.com/apache/arrow-go/v17>
> >> >> > > <http://github.com/apache/arrow-go/v17>
> >> >> > > > <http://github.com/apache/arrow-go/v17> or something to that
> >> end? Or
> >> >> > > > do we restart back at v1 (which I think would be confusing)?
> >> >> > > >
> >> >> > > > --Matt
> >> >> > > >
> >> >> > > > [1]: https://pkg.go.dev/github.com/apache/arrow/go/arrow
> >> >> > > >
> >> >> > > > On Thu, Jul 18, 2024 at 7:33 AM Antoine Pitrou <
> >> antoine@python.org>
> >> >> > > wrote:
> >> >> > > >
> >> >> > > > >
> >> >> > > > > Hi Kou,
> >> >> > > > >
> >> >> > > > > Le 18/07/2024 à 11:33, Sutou Kouhei a écrit :
> >> >> > > > > >
> >> >> > > > > > Here is my idea how to proceed this:
> >> >> > > > > >
> >> >> > > > > > 1. Extract go/ in apache/arrow to apache/arrow-go like
> >> >> > > > > > apache/arrow-rs
> >> >> > > > > > * Filter go/ related commits from apache/arrow and
> create
> >> >> > > > > > apache/arrow-go with them like we did for
> >> apache/arrow-rs
> >> >> > > > > > * Remove go/ related codes from apache/arrow
> >> >> > > > > > 2. Prepare integration test CI like apache/arrow-rs does:
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >>
> https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml
> >> >> > > > > > 3. Prepare release script based on apache/arrow-julia,
> >> >> > > > > > apache/arrow-adbc and/or
> >> apache/arrow-flight-sql-postgresql
> >> >> > > > >
> >> >> > > > > I think this is a good idea, but I'm not part of the Go
> >> maintainers.
> >> >> > > > >
> >> >> > > > > > Cons of this idea:
> >> >> > > > > >
> >> >> > > > > > * This is a backward incompatible change
> >> >> > > > > > * Users need to change their "import" to
> >> >> > > > > > "github.com/apache/arrow-go/arrow" from
> >> >> > > > > > "github.com/apache/arrow/go/arrow"
> >> >> > > > >
> >> >> > > > > Is there no way to leave some kind of alias or redirection in
> >> the
> >> >> > > > > apache/arrow repository?
> >> >> > > > >
> >> >> > > > > Regards
> >> >> > > > >
> >> >> > > > > Antoine.
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >>
>
Re: [DISCUSS] Split Go release process
Posted by Sutou Kouhei <ko...@clear-code.com>.
Hi,
Sorry for not working on this.
Thanks for sharing the standard docs! I've read it and
related docs.
Here is the summary I learned in this thread and the
standard docs:
1. We're using "github.com/apache/arrow/go/v${VERSION}" such
as "github.com/apache/arrow/go/v17" as our module name
* https://pkg.go.dev/github.com/apache/arrow/go/v17/arrow
* Including the version number part ("v${VERSION}") is
important
* Users can avoid unexpected backward incompatibility by
this style
2. We used to use "github.com/apache/arrow/go" as our module
name in v5 or earlier
* https://pkg.go.dev/github.com/apache/arrow/go/arrow
* 133 modules still use this
3. We want to avoid user side changes as much as possible
* As 2. shows, users may keep using old version if there
is any change is required
4. The current users need to change Apache Arrow Go's import
path to "github.com/apache/arrow/go/v${VERSION + 1}" when
they want to upgrade Apache Arrow Go
* We don't want to require more changes than "changing
import path" for users as mentioned in 3.
5. We can't provide backward compatible module name such as
"github.com/apache/arrow/go/v18" for
"github.com/apache/arrow-go/v18"
* Go doesn't provide the feature
6. We want to keep "v${VERSION}" in our module name even if
we split Apache Arrow Go to apache/arrow-go
* It's for avoiding unexpected backward incompatibility
in users' projects
Based on 6., users need to change their import paths on
upgrade whether we keep using apache/arrow or we use new
apache/arrow-go.
If we use new apache/arrow-go, we will be able to reduce
maintenance cost for apache/arrow (e.g. we can remove Go
related scripts, CI jobs and so on from apache/arrow). Let's
use apache/arrow-go.
If nobody objects splitting Apache Arrow Go to
apache/arrow-go in this week, I'll start working on this
next week. (Do we need a vote for this?)
Thanks,
--
kou
In <CA...@mail.gmail.com>
"Re: [DISCUSS] Split Go release process" on Mon, 22 Jul 2024 20:47:57 -0400,
Matt Topol <zo...@gmail.com> wrote:
> Hey Kou,
>
> https://go.dev/doc/modules/release-workflow is the standard docs for
> developing module versioning and publishing with Go.
>
> There isn't really a way to alias an import path to a different git repo
> because it uses the GitHub URL itself as the import path.
>
> But it does seem like people seem to prefer the idea of shifting the Go
> implementation to its own repository. I'd still push for us to include the
> major version number in the import path, and since we'll have fewer major
> releases and more minor releases, users shouldn't have to update their
> import paths as frequently.
>
> --Matt
>
> On Mon, Jul 22, 2024, 8:37 PM Sutou Kouhei <ko...@clear-code.com> wrote:
>
>> Hi,
>>
>> > Kou, is your plan also counting on moving the
>> > specific nightlies there and removing them from the main repo?
>>
>> Yes. I should have mentioned it explicitly.
>>
>> We will remove most Go related CI jobs from apache/arrow. We
>> will keep Go in integration test CI jobs like we do for
>> apache/arrow-rs.
>>
>>
>> Thanks,
>> --
>> kou
>>
>> In <CA...@mail.gmail.com>
>> "Re: [DISCUSS] Split Go release process" on Fri, 19 Jul 2024 17:14:25
>> +0200,
>> Raúl Cumplido <ra...@gmail.com> wrote:
>>
>> > Hi,
>> >
>> > The conversation around more frequent minor releases and version split
>> > per component has been a long one.
>> >
>> > I am in favour of these changes for the Go implementation because we
>> > have several maintainers.
>> >
>> > It might be difficult to release other implementations that do not
>> > have the same amount of maintainers. I am not sure what our plan is if
>> > one of the split implementations has less maintainers and there's a
>> > requirement for a release (i.e. security fix) but that might be
>> > something to consider in the future.
>> >
>> >> I would defer to Raul and Jacob to corroborate this, but because
>> >> changes to the CI configuration and release verification scripts don't
>> >> affect other implementations, I have been able to maintain that
>> >> infrastructure myself without too much effort and don't have to lean
>> >> on them for anything except reviews.
>> >
>> > I think releasing and maintaining release scripts / verifications per
>> > component is much easier than for the mono repo. We currently have
>> > over 200 nightly CI jobs in the mono repo that are required to pass
>> > before releasing. Moving some of those to its own repo helps
>> > maintainability. Kou, is your plan also counting on moving the
>> > specific nightlies there and removing them from the main repo?
>> >
>> > I would be in favour of doing a new major release (v18) once the repo
>> > and the changes are in-place to update the import path to something
>> > like:
>> > github.com/apache/arrow-go/v18
>> >
>> > This would avoid confusion with previous releases. We can then follow
>> > up with patch/minor/major as required.
>> >
>> > I am also happy to help with the releases and infrastructure if
>> > necessary as I've done with the main Arrow one (I can also help on
>> > nanoarrow, adbc if necessary).
>> >
>> > Kind regards,
>> > Raul
>> >
>> >
>> >
>> >>
>> >> [1] https://github.com/apache/arrow-nanoarrow/pull/557
>> >>
>> >> On Thu, Jul 18, 2024 at 7:53 PM Matt Topol <zo...@gmail.com>
>> wrote:
>> >> >
>> >> > Part of the goal of splitting out the release processes is that we'd
>> be
>> >> > able to do minor version releases more frequently instead of major
>> version
>> >> > releases.
>> >> >
>> >> > The general convention in the Go community is to include a major
>> version
>> >> > "v#" in the import path for all major versions past v1 so that if
>> there's a
>> >> > breaking change, it's explicit and prevents potential issues from
>> different
>> >> > major versions being used simultaneously. Being able to do minor
>> version
>> >> > releases more frequently would lead to not having to change the import
>> >> > paths every 3-6 months, but only if we actually do a breaking change.
>> >> >
>> >> > On Thu, Jul 18, 2024, 3:55 PM George Godik <gg...@gmail.com> wrote:
>> >> >
>> >> > > > If we shift the Go lib to a new/different import
>> >> > > path we'll end up with the same problem where people will rely on
>> older
>> >> > > versions and an incorrect path.
>> >> > >
>> >> > > Major version upgrades already require changing the import paths by
>> >> > > increasing the version. The proposed change would require everyone
>> to go
>> >> > > through a similar process one last time.
>> >> > >
>> >> > > > More to the point, there would be the question of whether or not
>> we
>> >> > > should port over the same major version
>> >> > > number, i.e. `github.com/apache/arrow-go/v17`
>> <http://github.com/apache/arrow-go/v17>
>> >> > > <http://github.com/apache/arrow-go/v17>
>> >> > > <http://github.com/apache/arrow-go/v17> or something to that end?
>> Or
>> >> > > do we restart back at v1 (which I think would be confusing)?
>> >> > >
>> >> > > My vote - for whatever it's worth - would be to do away with the
>> >> > > version-in-path naming convention and relying on the go
>> version/package
>> >> > > system for major upgrades.
>> >> > >
>> >> > > Benefits: I don't have to change import paths every 3-6months
>> >> > >
>> >> > > On Thu, Jul 18, 2024 at 3:34 PM Matt Topol <zo...@gmail.com>
>> wrote:
>> >> > >
>> >> > > > My thoughts:
>> >> > > >
>> >> > > > > * Go doesn't depend on other components such as C++
>> >> > > > > * Go has some active PMC member (Matt) and committer (Joel)
>> >> > > > > * Could you become a release manager for Go?
>> >> > > >
>> >> > > > I'd happily be the release manager for the Go implementation.
>> >> > > >
>> >> > > > > Here is my idea how to proceed this:
>> >> > > > >
>> >> > > > > 1. Extract go/ in apache/arrow to apache/arrow-go like
>> >> > > > > apache/arrow-rs
>> >> > > > > * Filter go/ related commits from apache/arrow and create
>> >> > > > > apache/arrow-go with them like we did for apache/arrow-rs
>> >> > > > > * Remove go/ related codes from apache/arrow
>> >> > > > > 2. Prepare integration test CI like apache/arrow-rs does:
>> >> > > > >
>> >> > > >
>> >> > > >
>> >> > >
>> https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml
>> >> > > > > 3. Prepare release script based on apache/arrow-julia,
>> >> > > > > apache/arrow-adbc and/or apache/arrow-flight-sql-postgresql
>> >> > > >
>> >> > > > Personally I would prefer that we do not extract it to its own
>> separate
>> >> > > > repository purely because I don't want to change the import path
>> for
>> >> > > users
>> >> > > > again. We already have this issue from before we introduced the
>> major
>> >> > > > version into the import path and shifted it up to allow for the
>> Parquet
>> >> > > lib
>> >> > > > in the same repository. If you look at [1] you see that there's
>> still
>> >> > > over
>> >> > > > 100 projects that never upgraded to v6 or higher because they are
>> still
>> >> > > > using the old import path. If we shift the Go lib to a
>> new/different
>> >> > > import
>> >> > > > path we'll end up with the same problem where people will rely on
>> older
>> >> > > > versions and an incorrect path.
>> >> > > >
>> >> > > > If we as a community decide that splitting out the
>> implementations all
>> >> > > into
>> >> > > > separate repositories is the best way forward, I won't hold it up
>> by
>> >> > > > strictly hammering on this. I'm just concerned about the
>> realities and
>> >> > > > difficulties of communicating the import path change, ensuring we
>> don't
>> >> > > > break any consumers, and ensuring that users still end up being
>> able to
>> >> > > > upgrade easily.
>> >> > > >
>> >> > > > > The import path could be "github.com/apache/arrow-go" instead
>> of "
>> >> > > > github.com/apache/arrow-go/arrow". Since go will allow users to
>> use
>> >> > > > `arrow.Abc` directly if user imports `github.com/apache/arrow-go`
>> <http://github.com/apache/arrow-go>
>> >> > > <http://github.com/apache/arrow-go>
>> >> > > > <http://github.com/apache/arrow-go>
>> >> > > > <http://github.com/apache/arrow-go>.
>> >> > > >
>> >> > > > The import path would still have to be `
>> >> > > github.com/apache/arrow-go/arrow`
>> <http://github.com/apache/arrow-go/arrow>
>> >> > > <http://github.com/apache/arrow-go/arrow>
>> >> > > > <http://github.com/apache/arrow-go/arrow>
>> >> > > > since it would also contain the parquet implementation in `
>> >> > > > github.com/apache/arrow-go/parquet`
>> <http://github.com/apache/arrow-go/parquet>
>> >> > > <http://github.com/apache/arrow-go/parquet>
>> >> > > > <http://github.com/apache/arrow-go/parquet>. More to the point,
>> there
>> >> > > > would be the
>> >> > > > question of whether or not we should port over the same major
>> version
>> >> > > > number, i.e. `github.com/apache/arrow-go/v17`
>> <http://github.com/apache/arrow-go/v17>
>> >> > > <http://github.com/apache/arrow-go/v17>
>> >> > > > <http://github.com/apache/arrow-go/v17> or something to that
>> end? Or
>> >> > > > do we restart back at v1 (which I think would be confusing)?
>> >> > > >
>> >> > > > --Matt
>> >> > > >
>> >> > > > [1]: https://pkg.go.dev/github.com/apache/arrow/go/arrow
>> >> > > >
>> >> > > > On Thu, Jul 18, 2024 at 7:33 AM Antoine Pitrou <
>> antoine@python.org>
>> >> > > wrote:
>> >> > > >
>> >> > > > >
>> >> > > > > Hi Kou,
>> >> > > > >
>> >> > > > > Le 18/07/2024 à 11:33, Sutou Kouhei a écrit :
>> >> > > > > >
>> >> > > > > > Here is my idea how to proceed this:
>> >> > > > > >
>> >> > > > > > 1. Extract go/ in apache/arrow to apache/arrow-go like
>> >> > > > > > apache/arrow-rs
>> >> > > > > > * Filter go/ related commits from apache/arrow and create
>> >> > > > > > apache/arrow-go with them like we did for
>> apache/arrow-rs
>> >> > > > > > * Remove go/ related codes from apache/arrow
>> >> > > > > > 2. Prepare integration test CI like apache/arrow-rs does:
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml
>> >> > > > > > 3. Prepare release script based on apache/arrow-julia,
>> >> > > > > > apache/arrow-adbc and/or
>> apache/arrow-flight-sql-postgresql
>> >> > > > >
>> >> > > > > I think this is a good idea, but I'm not part of the Go
>> maintainers.
>> >> > > > >
>> >> > > > > > Cons of this idea:
>> >> > > > > >
>> >> > > > > > * This is a backward incompatible change
>> >> > > > > > * Users need to change their "import" to
>> >> > > > > > "github.com/apache/arrow-go/arrow" from
>> >> > > > > > "github.com/apache/arrow/go/arrow"
>> >> > > > >
>> >> > > > > Is there no way to leave some kind of alias or redirection in
>> the
>> >> > > > > apache/arrow repository?
>> >> > > > >
>> >> > > > > Regards
>> >> > > > >
>> >> > > > > Antoine.
>> >> > > > >
>> >> > > >
>> >> > >
>>
Re: [DISCUSS] Split Go release process
Posted by Matt Topol <zo...@gmail.com>.
Hey Kou,
https://go.dev/doc/modules/release-workflow is the standard docs for
developing module versioning and publishing with Go.
There isn't really a way to alias an import path to a different git repo
because it uses the GitHub URL itself as the import path.
But it does seem like people seem to prefer the idea of shifting the Go
implementation to its own repository. I'd still push for us to include the
major version number in the import path, and since we'll have fewer major
releases and more minor releases, users shouldn't have to update their
import paths as frequently.
--Matt
On Mon, Jul 22, 2024, 8:37 PM Sutou Kouhei <ko...@clear-code.com> wrote:
> Hi,
>
> > Kou, is your plan also counting on moving the
> > specific nightlies there and removing them from the main repo?
>
> Yes. I should have mentioned it explicitly.
>
> We will remove most Go related CI jobs from apache/arrow. We
> will keep Go in integration test CI jobs like we do for
> apache/arrow-rs.
>
>
> Thanks,
> --
> kou
>
> In <CA...@mail.gmail.com>
> "Re: [DISCUSS] Split Go release process" on Fri, 19 Jul 2024 17:14:25
> +0200,
> Raúl Cumplido <ra...@gmail.com> wrote:
>
> > Hi,
> >
> > The conversation around more frequent minor releases and version split
> > per component has been a long one.
> >
> > I am in favour of these changes for the Go implementation because we
> > have several maintainers.
> >
> > It might be difficult to release other implementations that do not
> > have the same amount of maintainers. I am not sure what our plan is if
> > one of the split implementations has less maintainers and there's a
> > requirement for a release (i.e. security fix) but that might be
> > something to consider in the future.
> >
> >> I would defer to Raul and Jacob to corroborate this, but because
> >> changes to the CI configuration and release verification scripts don't
> >> affect other implementations, I have been able to maintain that
> >> infrastructure myself without too much effort and don't have to lean
> >> on them for anything except reviews.
> >
> > I think releasing and maintaining release scripts / verifications per
> > component is much easier than for the mono repo. We currently have
> > over 200 nightly CI jobs in the mono repo that are required to pass
> > before releasing. Moving some of those to its own repo helps
> > maintainability. Kou, is your plan also counting on moving the
> > specific nightlies there and removing them from the main repo?
> >
> > I would be in favour of doing a new major release (v18) once the repo
> > and the changes are in-place to update the import path to something
> > like:
> > github.com/apache/arrow-go/v18
> >
> > This would avoid confusion with previous releases. We can then follow
> > up with patch/minor/major as required.
> >
> > I am also happy to help with the releases and infrastructure if
> > necessary as I've done with the main Arrow one (I can also help on
> > nanoarrow, adbc if necessary).
> >
> > Kind regards,
> > Raul
> >
> >
> >
> >>
> >> [1] https://github.com/apache/arrow-nanoarrow/pull/557
> >>
> >> On Thu, Jul 18, 2024 at 7:53 PM Matt Topol <zo...@gmail.com>
> wrote:
> >> >
> >> > Part of the goal of splitting out the release processes is that we'd
> be
> >> > able to do minor version releases more frequently instead of major
> version
> >> > releases.
> >> >
> >> > The general convention in the Go community is to include a major
> version
> >> > "v#" in the import path for all major versions past v1 so that if
> there's a
> >> > breaking change, it's explicit and prevents potential issues from
> different
> >> > major versions being used simultaneously. Being able to do minor
> version
> >> > releases more frequently would lead to not having to change the import
> >> > paths every 3-6 months, but only if we actually do a breaking change.
> >> >
> >> > On Thu, Jul 18, 2024, 3:55 PM George Godik <gg...@gmail.com> wrote:
> >> >
> >> > > > If we shift the Go lib to a new/different import
> >> > > path we'll end up with the same problem where people will rely on
> older
> >> > > versions and an incorrect path.
> >> > >
> >> > > Major version upgrades already require changing the import paths by
> >> > > increasing the version. The proposed change would require everyone
> to go
> >> > > through a similar process one last time.
> >> > >
> >> > > > More to the point, there would be the question of whether or not
> we
> >> > > should port over the same major version
> >> > > number, i.e. `github.com/apache/arrow-go/v17`
> <http://github.com/apache/arrow-go/v17>
> >> > > <http://github.com/apache/arrow-go/v17>
> >> > > <http://github.com/apache/arrow-go/v17> or something to that end?
> Or
> >> > > do we restart back at v1 (which I think would be confusing)?
> >> > >
> >> > > My vote - for whatever it's worth - would be to do away with the
> >> > > version-in-path naming convention and relying on the go
> version/package
> >> > > system for major upgrades.
> >> > >
> >> > > Benefits: I don't have to change import paths every 3-6months
> >> > >
> >> > > On Thu, Jul 18, 2024 at 3:34 PM Matt Topol <zo...@gmail.com>
> wrote:
> >> > >
> >> > > > My thoughts:
> >> > > >
> >> > > > > * Go doesn't depend on other components such as C++
> >> > > > > * Go has some active PMC member (Matt) and committer (Joel)
> >> > > > > * Could you become a release manager for Go?
> >> > > >
> >> > > > I'd happily be the release manager for the Go implementation.
> >> > > >
> >> > > > > Here is my idea how to proceed this:
> >> > > > >
> >> > > > > 1. Extract go/ in apache/arrow to apache/arrow-go like
> >> > > > > apache/arrow-rs
> >> > > > > * Filter go/ related commits from apache/arrow and create
> >> > > > > apache/arrow-go with them like we did for apache/arrow-rs
> >> > > > > * Remove go/ related codes from apache/arrow
> >> > > > > 2. Prepare integration test CI like apache/arrow-rs does:
> >> > > > >
> >> > > >
> >> > > >
> >> > >
> https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml
> >> > > > > 3. Prepare release script based on apache/arrow-julia,
> >> > > > > apache/arrow-adbc and/or apache/arrow-flight-sql-postgresql
> >> > > >
> >> > > > Personally I would prefer that we do not extract it to its own
> separate
> >> > > > repository purely because I don't want to change the import path
> for
> >> > > users
> >> > > > again. We already have this issue from before we introduced the
> major
> >> > > > version into the import path and shifted it up to allow for the
> Parquet
> >> > > lib
> >> > > > in the same repository. If you look at [1] you see that there's
> still
> >> > > over
> >> > > > 100 projects that never upgraded to v6 or higher because they are
> still
> >> > > > using the old import path. If we shift the Go lib to a
> new/different
> >> > > import
> >> > > > path we'll end up with the same problem where people will rely on
> older
> >> > > > versions and an incorrect path.
> >> > > >
> >> > > > If we as a community decide that splitting out the
> implementations all
> >> > > into
> >> > > > separate repositories is the best way forward, I won't hold it up
> by
> >> > > > strictly hammering on this. I'm just concerned about the
> realities and
> >> > > > difficulties of communicating the import path change, ensuring we
> don't
> >> > > > break any consumers, and ensuring that users still end up being
> able to
> >> > > > upgrade easily.
> >> > > >
> >> > > > > The import path could be "github.com/apache/arrow-go" instead
> of "
> >> > > > github.com/apache/arrow-go/arrow". Since go will allow users to
> use
> >> > > > `arrow.Abc` directly if user imports `github.com/apache/arrow-go`
> <http://github.com/apache/arrow-go>
> >> > > <http://github.com/apache/arrow-go>
> >> > > > <http://github.com/apache/arrow-go>
> >> > > > <http://github.com/apache/arrow-go>.
> >> > > >
> >> > > > The import path would still have to be `
> >> > > github.com/apache/arrow-go/arrow`
> <http://github.com/apache/arrow-go/arrow>
> >> > > <http://github.com/apache/arrow-go/arrow>
> >> > > > <http://github.com/apache/arrow-go/arrow>
> >> > > > since it would also contain the parquet implementation in `
> >> > > > github.com/apache/arrow-go/parquet`
> <http://github.com/apache/arrow-go/parquet>
> >> > > <http://github.com/apache/arrow-go/parquet>
> >> > > > <http://github.com/apache/arrow-go/parquet>. More to the point,
> there
> >> > > > would be the
> >> > > > question of whether or not we should port over the same major
> version
> >> > > > number, i.e. `github.com/apache/arrow-go/v17`
> <http://github.com/apache/arrow-go/v17>
> >> > > <http://github.com/apache/arrow-go/v17>
> >> > > > <http://github.com/apache/arrow-go/v17> or something to that
> end? Or
> >> > > > do we restart back at v1 (which I think would be confusing)?
> >> > > >
> >> > > > --Matt
> >> > > >
> >> > > > [1]: https://pkg.go.dev/github.com/apache/arrow/go/arrow
> >> > > >
> >> > > > On Thu, Jul 18, 2024 at 7:33 AM Antoine Pitrou <
> antoine@python.org>
> >> > > wrote:
> >> > > >
> >> > > > >
> >> > > > > Hi Kou,
> >> > > > >
> >> > > > > Le 18/07/2024 à 11:33, Sutou Kouhei a écrit :
> >> > > > > >
> >> > > > > > Here is my idea how to proceed this:
> >> > > > > >
> >> > > > > > 1. Extract go/ in apache/arrow to apache/arrow-go like
> >> > > > > > apache/arrow-rs
> >> > > > > > * Filter go/ related commits from apache/arrow and create
> >> > > > > > apache/arrow-go with them like we did for
> apache/arrow-rs
> >> > > > > > * Remove go/ related codes from apache/arrow
> >> > > > > > 2. Prepare integration test CI like apache/arrow-rs does:
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml
> >> > > > > > 3. Prepare release script based on apache/arrow-julia,
> >> > > > > > apache/arrow-adbc and/or
> apache/arrow-flight-sql-postgresql
> >> > > > >
> >> > > > > I think this is a good idea, but I'm not part of the Go
> maintainers.
> >> > > > >
> >> > > > > > Cons of this idea:
> >> > > > > >
> >> > > > > > * This is a backward incompatible change
> >> > > > > > * Users need to change their "import" to
> >> > > > > > "github.com/apache/arrow-go/arrow" from
> >> > > > > > "github.com/apache/arrow/go/arrow"
> >> > > > >
> >> > > > > Is there no way to leave some kind of alias or redirection in
> the
> >> > > > > apache/arrow repository?
> >> > > > >
> >> > > > > Regards
> >> > > > >
> >> > > > > Antoine.
> >> > > > >
> >> > > >
> >> > >
>
Re: [DISCUSS] Split Go release process
Posted by Sutou Kouhei <ko...@clear-code.com>.
Hi,
> Kou, is your plan also counting on moving the
> specific nightlies there and removing them from the main repo?
Yes. I should have mentioned it explicitly.
We will remove most Go related CI jobs from apache/arrow. We
will keep Go in integration test CI jobs like we do for
apache/arrow-rs.
Thanks,
--
kou
In <CA...@mail.gmail.com>
"Re: [DISCUSS] Split Go release process" on Fri, 19 Jul 2024 17:14:25 +0200,
Raúl Cumplido <ra...@gmail.com> wrote:
> Hi,
>
> The conversation around more frequent minor releases and version split
> per component has been a long one.
>
> I am in favour of these changes for the Go implementation because we
> have several maintainers.
>
> It might be difficult to release other implementations that do not
> have the same amount of maintainers. I am not sure what our plan is if
> one of the split implementations has less maintainers and there's a
> requirement for a release (i.e. security fix) but that might be
> something to consider in the future.
>
>> I would defer to Raul and Jacob to corroborate this, but because
>> changes to the CI configuration and release verification scripts don't
>> affect other implementations, I have been able to maintain that
>> infrastructure myself without too much effort and don't have to lean
>> on them for anything except reviews.
>
> I think releasing and maintaining release scripts / verifications per
> component is much easier than for the mono repo. We currently have
> over 200 nightly CI jobs in the mono repo that are required to pass
> before releasing. Moving some of those to its own repo helps
> maintainability. Kou, is your plan also counting on moving the
> specific nightlies there and removing them from the main repo?
>
> I would be in favour of doing a new major release (v18) once the repo
> and the changes are in-place to update the import path to something
> like:
> github.com/apache/arrow-go/v18
>
> This would avoid confusion with previous releases. We can then follow
> up with patch/minor/major as required.
>
> I am also happy to help with the releases and infrastructure if
> necessary as I've done with the main Arrow one (I can also help on
> nanoarrow, adbc if necessary).
>
> Kind regards,
> Raul
>
>
>
>>
>> [1] https://github.com/apache/arrow-nanoarrow/pull/557
>>
>> On Thu, Jul 18, 2024 at 7:53 PM Matt Topol <zo...@gmail.com> wrote:
>> >
>> > Part of the goal of splitting out the release processes is that we'd be
>> > able to do minor version releases more frequently instead of major version
>> > releases.
>> >
>> > The general convention in the Go community is to include a major version
>> > "v#" in the import path for all major versions past v1 so that if there's a
>> > breaking change, it's explicit and prevents potential issues from different
>> > major versions being used simultaneously. Being able to do minor version
>> > releases more frequently would lead to not having to change the import
>> > paths every 3-6 months, but only if we actually do a breaking change.
>> >
>> > On Thu, Jul 18, 2024, 3:55 PM George Godik <gg...@gmail.com> wrote:
>> >
>> > > > If we shift the Go lib to a new/different import
>> > > path we'll end up with the same problem where people will rely on older
>> > > versions and an incorrect path.
>> > >
>> > > Major version upgrades already require changing the import paths by
>> > > increasing the version. The proposed change would require everyone to go
>> > > through a similar process one last time.
>> > >
>> > > > More to the point, there would be the question of whether or not we
>> > > should port over the same major version
>> > > number, i.e. `github.com/apache/arrow-go/v17`
>> > > <http://github.com/apache/arrow-go/v17>
>> > > <http://github.com/apache/arrow-go/v17> or something to that end? Or
>> > > do we restart back at v1 (which I think would be confusing)?
>> > >
>> > > My vote - for whatever it's worth - would be to do away with the
>> > > version-in-path naming convention and relying on the go version/package
>> > > system for major upgrades.
>> > >
>> > > Benefits: I don't have to change import paths every 3-6months
>> > >
>> > > On Thu, Jul 18, 2024 at 3:34 PM Matt Topol <zo...@gmail.com> wrote:
>> > >
>> > > > My thoughts:
>> > > >
>> > > > > * Go doesn't depend on other components such as C++
>> > > > > * Go has some active PMC member (Matt) and committer (Joel)
>> > > > > * Could you become a release manager for Go?
>> > > >
>> > > > I'd happily be the release manager for the Go implementation.
>> > > >
>> > > > > Here is my idea how to proceed this:
>> > > > >
>> > > > > 1. Extract go/ in apache/arrow to apache/arrow-go like
>> > > > > apache/arrow-rs
>> > > > > * Filter go/ related commits from apache/arrow and create
>> > > > > apache/arrow-go with them like we did for apache/arrow-rs
>> > > > > * Remove go/ related codes from apache/arrow
>> > > > > 2. Prepare integration test CI like apache/arrow-rs does:
>> > > > >
>> > > >
>> > > >
>> > > https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml
>> > > > > 3. Prepare release script based on apache/arrow-julia,
>> > > > > apache/arrow-adbc and/or apache/arrow-flight-sql-postgresql
>> > > >
>> > > > Personally I would prefer that we do not extract it to its own separate
>> > > > repository purely because I don't want to change the import path for
>> > > users
>> > > > again. We already have this issue from before we introduced the major
>> > > > version into the import path and shifted it up to allow for the Parquet
>> > > lib
>> > > > in the same repository. If you look at [1] you see that there's still
>> > > over
>> > > > 100 projects that never upgraded to v6 or higher because they are still
>> > > > using the old import path. If we shift the Go lib to a new/different
>> > > import
>> > > > path we'll end up with the same problem where people will rely on older
>> > > > versions and an incorrect path.
>> > > >
>> > > > If we as a community decide that splitting out the implementations all
>> > > into
>> > > > separate repositories is the best way forward, I won't hold it up by
>> > > > strictly hammering on this. I'm just concerned about the realities and
>> > > > difficulties of communicating the import path change, ensuring we don't
>> > > > break any consumers, and ensuring that users still end up being able to
>> > > > upgrade easily.
>> > > >
>> > > > > The import path could be "github.com/apache/arrow-go" instead of "
>> > > > github.com/apache/arrow-go/arrow". Since go will allow users to use
>> > > > `arrow.Abc` directly if user imports `github.com/apache/arrow-go`
>> > > <http://github.com/apache/arrow-go>
>> > > > <http://github.com/apache/arrow-go>
>> > > > <http://github.com/apache/arrow-go>.
>> > > >
>> > > > The import path would still have to be `
>> > > github.com/apache/arrow-go/arrow`
>> > > <http://github.com/apache/arrow-go/arrow>
>> > > > <http://github.com/apache/arrow-go/arrow>
>> > > > since it would also contain the parquet implementation in `
>> > > > github.com/apache/arrow-go/parquet`
>> > > <http://github.com/apache/arrow-go/parquet>
>> > > > <http://github.com/apache/arrow-go/parquet>. More to the point, there
>> > > > would be the
>> > > > question of whether or not we should port over the same major version
>> > > > number, i.e. `github.com/apache/arrow-go/v17`
>> > > <http://github.com/apache/arrow-go/v17>
>> > > > <http://github.com/apache/arrow-go/v17> or something to that end? Or
>> > > > do we restart back at v1 (which I think would be confusing)?
>> > > >
>> > > > --Matt
>> > > >
>> > > > [1]: https://pkg.go.dev/github.com/apache/arrow/go/arrow
>> > > >
>> > > > On Thu, Jul 18, 2024 at 7:33 AM Antoine Pitrou <an...@python.org>
>> > > wrote:
>> > > >
>> > > > >
>> > > > > Hi Kou,
>> > > > >
>> > > > > Le 18/07/2024 à 11:33, Sutou Kouhei a écrit :
>> > > > > >
>> > > > > > Here is my idea how to proceed this:
>> > > > > >
>> > > > > > 1. Extract go/ in apache/arrow to apache/arrow-go like
>> > > > > > apache/arrow-rs
>> > > > > > * Filter go/ related commits from apache/arrow and create
>> > > > > > apache/arrow-go with them like we did for apache/arrow-rs
>> > > > > > * Remove go/ related codes from apache/arrow
>> > > > > > 2. Prepare integration test CI like apache/arrow-rs does:
>> > > > > >
>> > > > >
>> > > >
>> > > https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml
>> > > > > > 3. Prepare release script based on apache/arrow-julia,
>> > > > > > apache/arrow-adbc and/or apache/arrow-flight-sql-postgresql
>> > > > >
>> > > > > I think this is a good idea, but I'm not part of the Go maintainers.
>> > > > >
>> > > > > > Cons of this idea:
>> > > > > >
>> > > > > > * This is a backward incompatible change
>> > > > > > * Users need to change their "import" to
>> > > > > > "github.com/apache/arrow-go/arrow" from
>> > > > > > "github.com/apache/arrow/go/arrow"
>> > > > >
>> > > > > Is there no way to leave some kind of alias or redirection in the
>> > > > > apache/arrow repository?
>> > > > >
>> > > > > Regards
>> > > > >
>> > > > > Antoine.
>> > > > >
>> > > >
>> > >
Re: [DISCUSS] Split Go release process
Posted by Raúl Cumplido <ra...@gmail.com>.
Hi,
The conversation around more frequent minor releases and version split
per component has been a long one.
I am in favour of these changes for the Go implementation because we
have several maintainers.
It might be difficult to release other implementations that do not
have the same amount of maintainers. I am not sure what our plan is if
one of the split implementations has less maintainers and there's a
requirement for a release (i.e. security fix) but that might be
something to consider in the future.
> I would defer to Raul and Jacob to corroborate this, but because
> changes to the CI configuration and release verification scripts don't
> affect other implementations, I have been able to maintain that
> infrastructure myself without too much effort and don't have to lean
> on them for anything except reviews.
I think releasing and maintaining release scripts / verifications per
component is much easier than for the mono repo. We currently have
over 200 nightly CI jobs in the mono repo that are required to pass
before releasing. Moving some of those to its own repo helps
maintainability. Kou, is your plan also counting on moving the
specific nightlies there and removing them from the main repo?
I would be in favour of doing a new major release (v18) once the repo
and the changes are in-place to update the import path to something
like:
github.com/apache/arrow-go/v18
This would avoid confusion with previous releases. We can then follow
up with patch/minor/major as required.
I am also happy to help with the releases and infrastructure if
necessary as I've done with the main Arrow one (I can also help on
nanoarrow, adbc if necessary).
Kind regards,
Raul
>
> [1] https://github.com/apache/arrow-nanoarrow/pull/557
>
> On Thu, Jul 18, 2024 at 7:53 PM Matt Topol <zo...@gmail.com> wrote:
> >
> > Part of the goal of splitting out the release processes is that we'd be
> > able to do minor version releases more frequently instead of major version
> > releases.
> >
> > The general convention in the Go community is to include a major version
> > "v#" in the import path for all major versions past v1 so that if there's a
> > breaking change, it's explicit and prevents potential issues from different
> > major versions being used simultaneously. Being able to do minor version
> > releases more frequently would lead to not having to change the import
> > paths every 3-6 months, but only if we actually do a breaking change.
> >
> > On Thu, Jul 18, 2024, 3:55 PM George Godik <gg...@gmail.com> wrote:
> >
> > > > If we shift the Go lib to a new/different import
> > > path we'll end up with the same problem where people will rely on older
> > > versions and an incorrect path.
> > >
> > > Major version upgrades already require changing the import paths by
> > > increasing the version. The proposed change would require everyone to go
> > > through a similar process one last time.
> > >
> > > > More to the point, there would be the question of whether or not we
> > > should port over the same major version
> > > number, i.e. `github.com/apache/arrow-go/v17`
> > > <http://github.com/apache/arrow-go/v17>
> > > <http://github.com/apache/arrow-go/v17> or something to that end? Or
> > > do we restart back at v1 (which I think would be confusing)?
> > >
> > > My vote - for whatever it's worth - would be to do away with the
> > > version-in-path naming convention and relying on the go version/package
> > > system for major upgrades.
> > >
> > > Benefits: I don't have to change import paths every 3-6months
> > >
> > > On Thu, Jul 18, 2024 at 3:34 PM Matt Topol <zo...@gmail.com> wrote:
> > >
> > > > My thoughts:
> > > >
> > > > > * Go doesn't depend on other components such as C++
> > > > > * Go has some active PMC member (Matt) and committer (Joel)
> > > > > * Could you become a release manager for Go?
> > > >
> > > > I'd happily be the release manager for the Go implementation.
> > > >
> > > > > Here is my idea how to proceed this:
> > > > >
> > > > > 1. Extract go/ in apache/arrow to apache/arrow-go like
> > > > > apache/arrow-rs
> > > > > * Filter go/ related commits from apache/arrow and create
> > > > > apache/arrow-go with them like we did for apache/arrow-rs
> > > > > * Remove go/ related codes from apache/arrow
> > > > > 2. Prepare integration test CI like apache/arrow-rs does:
> > > > >
> > > >
> > > >
> > > https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml
> > > > > 3. Prepare release script based on apache/arrow-julia,
> > > > > apache/arrow-adbc and/or apache/arrow-flight-sql-postgresql
> > > >
> > > > Personally I would prefer that we do not extract it to its own separate
> > > > repository purely because I don't want to change the import path for
> > > users
> > > > again. We already have this issue from before we introduced the major
> > > > version into the import path and shifted it up to allow for the Parquet
> > > lib
> > > > in the same repository. If you look at [1] you see that there's still
> > > over
> > > > 100 projects that never upgraded to v6 or higher because they are still
> > > > using the old import path. If we shift the Go lib to a new/different
> > > import
> > > > path we'll end up with the same problem where people will rely on older
> > > > versions and an incorrect path.
> > > >
> > > > If we as a community decide that splitting out the implementations all
> > > into
> > > > separate repositories is the best way forward, I won't hold it up by
> > > > strictly hammering on this. I'm just concerned about the realities and
> > > > difficulties of communicating the import path change, ensuring we don't
> > > > break any consumers, and ensuring that users still end up being able to
> > > > upgrade easily.
> > > >
> > > > > The import path could be "github.com/apache/arrow-go" instead of "
> > > > github.com/apache/arrow-go/arrow". Since go will allow users to use
> > > > `arrow.Abc` directly if user imports `github.com/apache/arrow-go`
> > > <http://github.com/apache/arrow-go>
> > > > <http://github.com/apache/arrow-go>
> > > > <http://github.com/apache/arrow-go>.
> > > >
> > > > The import path would still have to be `
> > > github.com/apache/arrow-go/arrow`
> > > <http://github.com/apache/arrow-go/arrow>
> > > > <http://github.com/apache/arrow-go/arrow>
> > > > since it would also contain the parquet implementation in `
> > > > github.com/apache/arrow-go/parquet`
> > > <http://github.com/apache/arrow-go/parquet>
> > > > <http://github.com/apache/arrow-go/parquet>. More to the point, there
> > > > would be the
> > > > question of whether or not we should port over the same major version
> > > > number, i.e. `github.com/apache/arrow-go/v17`
> > > <http://github.com/apache/arrow-go/v17>
> > > > <http://github.com/apache/arrow-go/v17> or something to that end? Or
> > > > do we restart back at v1 (which I think would be confusing)?
> > > >
> > > > --Matt
> > > >
> > > > [1]: https://pkg.go.dev/github.com/apache/arrow/go/arrow
> > > >
> > > > On Thu, Jul 18, 2024 at 7:33 AM Antoine Pitrou <an...@python.org>
> > > wrote:
> > > >
> > > > >
> > > > > Hi Kou,
> > > > >
> > > > > Le 18/07/2024 à 11:33, Sutou Kouhei a écrit :
> > > > > >
> > > > > > Here is my idea how to proceed this:
> > > > > >
> > > > > > 1. Extract go/ in apache/arrow to apache/arrow-go like
> > > > > > apache/arrow-rs
> > > > > > * Filter go/ related commits from apache/arrow and create
> > > > > > apache/arrow-go with them like we did for apache/arrow-rs
> > > > > > * Remove go/ related codes from apache/arrow
> > > > > > 2. Prepare integration test CI like apache/arrow-rs does:
> > > > > >
> > > > >
> > > >
> > > https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml
> > > > > > 3. Prepare release script based on apache/arrow-julia,
> > > > > > apache/arrow-adbc and/or apache/arrow-flight-sql-postgresql
> > > > >
> > > > > I think this is a good idea, but I'm not part of the Go maintainers.
> > > > >
> > > > > > Cons of this idea:
> > > > > >
> > > > > > * This is a backward incompatible change
> > > > > > * Users need to change their "import" to
> > > > > > "github.com/apache/arrow-go/arrow" from
> > > > > > "github.com/apache/arrow/go/arrow"
> > > > >
> > > > > Is there no way to leave some kind of alias or redirection in the
> > > > > apache/arrow repository?
> > > > >
> > > > > Regards
> > > > >
> > > > > Antoine.
> > > > >
> > > >
> > >
Re: [DISCUSS] Split Go release process
Posted by Dewey Dunnington <de...@voltrondata.com.INVALID>.
> Extract go/ in apache/arrow to apache/arrow-go like apache/arrow-rs
I am not go maintainer and would defer to Matt and Joel on whether or
not this is more work from their end, but it does seem like Go users
have been forced to change their import paths quite a lot already and
would probably be OK with this update given that it will mean they
will have to update it a lot less in the future. There is quite a lot
to draw on from arrow-rs, arrow-adbc, arrow-julia, and arrow-nanoarrow
such that the initial cost of the CI/release infrastructure is
probably not as substantial as it once was (I am happy to help). I
have very much enjoyed working in apache/arrow-nanoarrow and
apache/arrow-adbc and have heard anecdotal comments from contributors
that it has been enjoyable for them as well.
> Prepare integration test CI like apache/arrow-rs does:
> https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml
We just added something similar in arrow-nanoarrow based on the
arrow-rs setup as well [1].
> Prepare release script based on apache/arrow-julia,
> apache/arrow-adbc and/or apache/arrow-flight-sql-postgresql
I would defer to Raul and Jacob to corroborate this, but because
changes to the CI configuration and release verification scripts don't
affect other implementations, I have been able to maintain that
infrastructure myself without too much effort and don't have to lean
on them for anything except reviews.
[1] https://github.com/apache/arrow-nanoarrow/pull/557
On Thu, Jul 18, 2024 at 7:53 PM Matt Topol <zo...@gmail.com> wrote:
>
> Part of the goal of splitting out the release processes is that we'd be
> able to do minor version releases more frequently instead of major version
> releases.
>
> The general convention in the Go community is to include a major version
> "v#" in the import path for all major versions past v1 so that if there's a
> breaking change, it's explicit and prevents potential issues from different
> major versions being used simultaneously. Being able to do minor version
> releases more frequently would lead to not having to change the import
> paths every 3-6 months, but only if we actually do a breaking change.
>
> On Thu, Jul 18, 2024, 3:55 PM George Godik <gg...@gmail.com> wrote:
>
> > > If we shift the Go lib to a new/different import
> > path we'll end up with the same problem where people will rely on older
> > versions and an incorrect path.
> >
> > Major version upgrades already require changing the import paths by
> > increasing the version. The proposed change would require everyone to go
> > through a similar process one last time.
> >
> > > More to the point, there would be the question of whether or not we
> > should port over the same major version
> > number, i.e. `github.com/apache/arrow-go/v17`
> > <http://github.com/apache/arrow-go/v17>
> > <http://github.com/apache/arrow-go/v17> or something to that end? Or
> > do we restart back at v1 (which I think would be confusing)?
> >
> > My vote - for whatever it's worth - would be to do away with the
> > version-in-path naming convention and relying on the go version/package
> > system for major upgrades.
> >
> > Benefits: I don't have to change import paths every 3-6months
> >
> > On Thu, Jul 18, 2024 at 3:34 PM Matt Topol <zo...@gmail.com> wrote:
> >
> > > My thoughts:
> > >
> > > > * Go doesn't depend on other components such as C++
> > > > * Go has some active PMC member (Matt) and committer (Joel)
> > > > * Could you become a release manager for Go?
> > >
> > > I'd happily be the release manager for the Go implementation.
> > >
> > > > Here is my idea how to proceed this:
> > > >
> > > > 1. Extract go/ in apache/arrow to apache/arrow-go like
> > > > apache/arrow-rs
> > > > * Filter go/ related commits from apache/arrow and create
> > > > apache/arrow-go with them like we did for apache/arrow-rs
> > > > * Remove go/ related codes from apache/arrow
> > > > 2. Prepare integration test CI like apache/arrow-rs does:
> > > >
> > >
> > >
> > https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml
> > > > 3. Prepare release script based on apache/arrow-julia,
> > > > apache/arrow-adbc and/or apache/arrow-flight-sql-postgresql
> > >
> > > Personally I would prefer that we do not extract it to its own separate
> > > repository purely because I don't want to change the import path for
> > users
> > > again. We already have this issue from before we introduced the major
> > > version into the import path and shifted it up to allow for the Parquet
> > lib
> > > in the same repository. If you look at [1] you see that there's still
> > over
> > > 100 projects that never upgraded to v6 or higher because they are still
> > > using the old import path. If we shift the Go lib to a new/different
> > import
> > > path we'll end up with the same problem where people will rely on older
> > > versions and an incorrect path.
> > >
> > > If we as a community decide that splitting out the implementations all
> > into
> > > separate repositories is the best way forward, I won't hold it up by
> > > strictly hammering on this. I'm just concerned about the realities and
> > > difficulties of communicating the import path change, ensuring we don't
> > > break any consumers, and ensuring that users still end up being able to
> > > upgrade easily.
> > >
> > > > The import path could be "github.com/apache/arrow-go" instead of "
> > > github.com/apache/arrow-go/arrow". Since go will allow users to use
> > > `arrow.Abc` directly if user imports `github.com/apache/arrow-go`
> > <http://github.com/apache/arrow-go>
> > > <http://github.com/apache/arrow-go>
> > > <http://github.com/apache/arrow-go>.
> > >
> > > The import path would still have to be `
> > github.com/apache/arrow-go/arrow`
> > <http://github.com/apache/arrow-go/arrow>
> > > <http://github.com/apache/arrow-go/arrow>
> > > since it would also contain the parquet implementation in `
> > > github.com/apache/arrow-go/parquet`
> > <http://github.com/apache/arrow-go/parquet>
> > > <http://github.com/apache/arrow-go/parquet>. More to the point, there
> > > would be the
> > > question of whether or not we should port over the same major version
> > > number, i.e. `github.com/apache/arrow-go/v17`
> > <http://github.com/apache/arrow-go/v17>
> > > <http://github.com/apache/arrow-go/v17> or something to that end? Or
> > > do we restart back at v1 (which I think would be confusing)?
> > >
> > > --Matt
> > >
> > > [1]: https://pkg.go.dev/github.com/apache/arrow/go/arrow
> > >
> > > On Thu, Jul 18, 2024 at 7:33 AM Antoine Pitrou <an...@python.org>
> > wrote:
> > >
> > > >
> > > > Hi Kou,
> > > >
> > > > Le 18/07/2024 à 11:33, Sutou Kouhei a écrit :
> > > > >
> > > > > Here is my idea how to proceed this:
> > > > >
> > > > > 1. Extract go/ in apache/arrow to apache/arrow-go like
> > > > > apache/arrow-rs
> > > > > * Filter go/ related commits from apache/arrow and create
> > > > > apache/arrow-go with them like we did for apache/arrow-rs
> > > > > * Remove go/ related codes from apache/arrow
> > > > > 2. Prepare integration test CI like apache/arrow-rs does:
> > > > >
> > > >
> > >
> > https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml
> > > > > 3. Prepare release script based on apache/arrow-julia,
> > > > > apache/arrow-adbc and/or apache/arrow-flight-sql-postgresql
> > > >
> > > > I think this is a good idea, but I'm not part of the Go maintainers.
> > > >
> > > > > Cons of this idea:
> > > > >
> > > > > * This is a backward incompatible change
> > > > > * Users need to change their "import" to
> > > > > "github.com/apache/arrow-go/arrow" from
> > > > > "github.com/apache/arrow/go/arrow"
> > > >
> > > > Is there no way to leave some kind of alias or redirection in the
> > > > apache/arrow repository?
> > > >
> > > > Regards
> > > >
> > > > Antoine.
> > > >
> > >
> >
Re: [DISCUSS] Split Go release process
Posted by Matt Topol <zo...@gmail.com>.
Part of the goal of splitting out the release processes is that we'd be
able to do minor version releases more frequently instead of major version
releases.
The general convention in the Go community is to include a major version
"v#" in the import path for all major versions past v1 so that if there's a
breaking change, it's explicit and prevents potential issues from different
major versions being used simultaneously. Being able to do minor version
releases more frequently would lead to not having to change the import
paths every 3-6 months, but only if we actually do a breaking change.
On Thu, Jul 18, 2024, 3:55 PM George Godik <gg...@gmail.com> wrote:
> > If we shift the Go lib to a new/different import
> path we'll end up with the same problem where people will rely on older
> versions and an incorrect path.
>
> Major version upgrades already require changing the import paths by
> increasing the version. The proposed change would require everyone to go
> through a similar process one last time.
>
> > More to the point, there would be the question of whether or not we
> should port over the same major version
> number, i.e. `github.com/apache/arrow-go/v17`
> <http://github.com/apache/arrow-go/v17>
> <http://github.com/apache/arrow-go/v17> or something to that end? Or
> do we restart back at v1 (which I think would be confusing)?
>
> My vote - for whatever it's worth - would be to do away with the
> version-in-path naming convention and relying on the go version/package
> system for major upgrades.
>
> Benefits: I don't have to change import paths every 3-6months
>
> On Thu, Jul 18, 2024 at 3:34 PM Matt Topol <zo...@gmail.com> wrote:
>
> > My thoughts:
> >
> > > * Go doesn't depend on other components such as C++
> > > * Go has some active PMC member (Matt) and committer (Joel)
> > > * Could you become a release manager for Go?
> >
> > I'd happily be the release manager for the Go implementation.
> >
> > > Here is my idea how to proceed this:
> > >
> > > 1. Extract go/ in apache/arrow to apache/arrow-go like
> > > apache/arrow-rs
> > > * Filter go/ related commits from apache/arrow and create
> > > apache/arrow-go with them like we did for apache/arrow-rs
> > > * Remove go/ related codes from apache/arrow
> > > 2. Prepare integration test CI like apache/arrow-rs does:
> > >
> >
> >
> https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml
> > > 3. Prepare release script based on apache/arrow-julia,
> > > apache/arrow-adbc and/or apache/arrow-flight-sql-postgresql
> >
> > Personally I would prefer that we do not extract it to its own separate
> > repository purely because I don't want to change the import path for
> users
> > again. We already have this issue from before we introduced the major
> > version into the import path and shifted it up to allow for the Parquet
> lib
> > in the same repository. If you look at [1] you see that there's still
> over
> > 100 projects that never upgraded to v6 or higher because they are still
> > using the old import path. If we shift the Go lib to a new/different
> import
> > path we'll end up with the same problem where people will rely on older
> > versions and an incorrect path.
> >
> > If we as a community decide that splitting out the implementations all
> into
> > separate repositories is the best way forward, I won't hold it up by
> > strictly hammering on this. I'm just concerned about the realities and
> > difficulties of communicating the import path change, ensuring we don't
> > break any consumers, and ensuring that users still end up being able to
> > upgrade easily.
> >
> > > The import path could be "github.com/apache/arrow-go" instead of "
> > github.com/apache/arrow-go/arrow". Since go will allow users to use
> > `arrow.Abc` directly if user imports `github.com/apache/arrow-go`
> <http://github.com/apache/arrow-go>
> > <http://github.com/apache/arrow-go>
> > <http://github.com/apache/arrow-go>.
> >
> > The import path would still have to be `
> github.com/apache/arrow-go/arrow`
> <http://github.com/apache/arrow-go/arrow>
> > <http://github.com/apache/arrow-go/arrow>
> > since it would also contain the parquet implementation in `
> > github.com/apache/arrow-go/parquet`
> <http://github.com/apache/arrow-go/parquet>
> > <http://github.com/apache/arrow-go/parquet>. More to the point, there
> > would be the
> > question of whether or not we should port over the same major version
> > number, i.e. `github.com/apache/arrow-go/v17`
> <http://github.com/apache/arrow-go/v17>
> > <http://github.com/apache/arrow-go/v17> or something to that end? Or
> > do we restart back at v1 (which I think would be confusing)?
> >
> > --Matt
> >
> > [1]: https://pkg.go.dev/github.com/apache/arrow/go/arrow
> >
> > On Thu, Jul 18, 2024 at 7:33 AM Antoine Pitrou <an...@python.org>
> wrote:
> >
> > >
> > > Hi Kou,
> > >
> > > Le 18/07/2024 à 11:33, Sutou Kouhei a écrit :
> > > >
> > > > Here is my idea how to proceed this:
> > > >
> > > > 1. Extract go/ in apache/arrow to apache/arrow-go like
> > > > apache/arrow-rs
> > > > * Filter go/ related commits from apache/arrow and create
> > > > apache/arrow-go with them like we did for apache/arrow-rs
> > > > * Remove go/ related codes from apache/arrow
> > > > 2. Prepare integration test CI like apache/arrow-rs does:
> > > >
> > >
> >
> https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml
> > > > 3. Prepare release script based on apache/arrow-julia,
> > > > apache/arrow-adbc and/or apache/arrow-flight-sql-postgresql
> > >
> > > I think this is a good idea, but I'm not part of the Go maintainers.
> > >
> > > > Cons of this idea:
> > > >
> > > > * This is a backward incompatible change
> > > > * Users need to change their "import" to
> > > > "github.com/apache/arrow-go/arrow" from
> > > > "github.com/apache/arrow/go/arrow"
> > >
> > > Is there no way to leave some kind of alias or redirection in the
> > > apache/arrow repository?
> > >
> > > Regards
> > >
> > > Antoine.
> > >
> >
>
Re: [DISCUSS] Split Go release process
Posted by George Godik <gg...@gmail.com>.
> If we shift the Go lib to a new/different import
path we'll end up with the same problem where people will rely on older
versions and an incorrect path.
Major version upgrades already require changing the import paths by
increasing the version. The proposed change would require everyone to go
through a similar process one last time.
> More to the point, there would be the question of whether or not we
should port over the same major version
number, i.e. `github.com/apache/arrow-go/v17`
<http://github.com/apache/arrow-go/v17> or something to that end? Or
do we restart back at v1 (which I think would be confusing)?
My vote - for whatever it's worth - would be to do away with the
version-in-path naming convention and relying on the go version/package
system for major upgrades.
Benefits: I don't have to change import paths every 3-6months
On Thu, Jul 18, 2024 at 3:34 PM Matt Topol <zo...@gmail.com> wrote:
> My thoughts:
>
> > * Go doesn't depend on other components such as C++
> > * Go has some active PMC member (Matt) and committer (Joel)
> > * Could you become a release manager for Go?
>
> I'd happily be the release manager for the Go implementation.
>
> > Here is my idea how to proceed this:
> >
> > 1. Extract go/ in apache/arrow to apache/arrow-go like
> > apache/arrow-rs
> > * Filter go/ related commits from apache/arrow and create
> > apache/arrow-go with them like we did for apache/arrow-rs
> > * Remove go/ related codes from apache/arrow
> > 2. Prepare integration test CI like apache/arrow-rs does:
> >
>
> https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml
> > 3. Prepare release script based on apache/arrow-julia,
> > apache/arrow-adbc and/or apache/arrow-flight-sql-postgresql
>
> Personally I would prefer that we do not extract it to its own separate
> repository purely because I don't want to change the import path for users
> again. We already have this issue from before we introduced the major
> version into the import path and shifted it up to allow for the Parquet lib
> in the same repository. If you look at [1] you see that there's still over
> 100 projects that never upgraded to v6 or higher because they are still
> using the old import path. If we shift the Go lib to a new/different import
> path we'll end up with the same problem where people will rely on older
> versions and an incorrect path.
>
> If we as a community decide that splitting out the implementations all into
> separate repositories is the best way forward, I won't hold it up by
> strictly hammering on this. I'm just concerned about the realities and
> difficulties of communicating the import path change, ensuring we don't
> break any consumers, and ensuring that users still end up being able to
> upgrade easily.
>
> > The import path could be "github.com/apache/arrow-go" instead of "
> github.com/apache/arrow-go/arrow". Since go will allow users to use
> `arrow.Abc` directly if user imports `github.com/apache/arrow-go`
> <http://github.com/apache/arrow-go>
> <http://github.com/apache/arrow-go>.
>
> The import path would still have to be `github.com/apache/arrow-go/arrow`
> <http://github.com/apache/arrow-go/arrow>
> since it would also contain the parquet implementation in `
> github.com/apache/arrow-go/parquet`
> <http://github.com/apache/arrow-go/parquet>. More to the point, there
> would be the
> question of whether or not we should port over the same major version
> number, i.e. `github.com/apache/arrow-go/v17`
> <http://github.com/apache/arrow-go/v17> or something to that end? Or
> do we restart back at v1 (which I think would be confusing)?
>
> --Matt
>
> [1]: https://pkg.go.dev/github.com/apache/arrow/go/arrow
>
> On Thu, Jul 18, 2024 at 7:33 AM Antoine Pitrou <an...@python.org> wrote:
>
> >
> > Hi Kou,
> >
> > Le 18/07/2024 à 11:33, Sutou Kouhei a écrit :
> > >
> > > Here is my idea how to proceed this:
> > >
> > > 1. Extract go/ in apache/arrow to apache/arrow-go like
> > > apache/arrow-rs
> > > * Filter go/ related commits from apache/arrow and create
> > > apache/arrow-go with them like we did for apache/arrow-rs
> > > * Remove go/ related codes from apache/arrow
> > > 2. Prepare integration test CI like apache/arrow-rs does:
> > >
> >
> https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml
> > > 3. Prepare release script based on apache/arrow-julia,
> > > apache/arrow-adbc and/or apache/arrow-flight-sql-postgresql
> >
> > I think this is a good idea, but I'm not part of the Go maintainers.
> >
> > > Cons of this idea:
> > >
> > > * This is a backward incompatible change
> > > * Users need to change their "import" to
> > > "github.com/apache/arrow-go/arrow" from
> > > "github.com/apache/arrow/go/arrow"
> >
> > Is there no way to leave some kind of alias or redirection in the
> > apache/arrow repository?
> >
> > Regards
> >
> > Antoine.
> >
>
Re: [DISCUSS] Split Go release process
Posted by Matt Topol <zo...@gmail.com>.
My thoughts:
> * Go doesn't depend on other components such as C++
> * Go has some active PMC member (Matt) and committer (Joel)
> * Could you become a release manager for Go?
I'd happily be the release manager for the Go implementation.
> Here is my idea how to proceed this:
>
> 1. Extract go/ in apache/arrow to apache/arrow-go like
> apache/arrow-rs
> * Filter go/ related commits from apache/arrow and create
> apache/arrow-go with them like we did for apache/arrow-rs
> * Remove go/ related codes from apache/arrow
> 2. Prepare integration test CI like apache/arrow-rs does:
>
https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml
> 3. Prepare release script based on apache/arrow-julia,
> apache/arrow-adbc and/or apache/arrow-flight-sql-postgresql
Personally I would prefer that we do not extract it to its own separate
repository purely because I don't want to change the import path for users
again. We already have this issue from before we introduced the major
version into the import path and shifted it up to allow for the Parquet lib
in the same repository. If you look at [1] you see that there's still over
100 projects that never upgraded to v6 or higher because they are still
using the old import path. If we shift the Go lib to a new/different import
path we'll end up with the same problem where people will rely on older
versions and an incorrect path.
If we as a community decide that splitting out the implementations all into
separate repositories is the best way forward, I won't hold it up by
strictly hammering on this. I'm just concerned about the realities and
difficulties of communicating the import path change, ensuring we don't
break any consumers, and ensuring that users still end up being able to
upgrade easily.
> The import path could be "github.com/apache/arrow-go" instead of "
github.com/apache/arrow-go/arrow". Since go will allow users to use
`arrow.Abc` directly if user imports `github.com/apache/arrow-go`
<http://github.com/apache/arrow-go>.
The import path would still have to be `github.com/apache/arrow-go/arrow`
since it would also contain the parquet implementation in `
github.com/apache/arrow-go/parquet`. More to the point, there would be the
question of whether or not we should port over the same major version
number, i.e. `github.com/apache/arrow-go/v17` or something to that end? Or
do we restart back at v1 (which I think would be confusing)?
--Matt
[1]: https://pkg.go.dev/github.com/apache/arrow/go/arrow
On Thu, Jul 18, 2024 at 7:33 AM Antoine Pitrou <an...@python.org> wrote:
>
> Hi Kou,
>
> Le 18/07/2024 à 11:33, Sutou Kouhei a écrit :
> >
> > Here is my idea how to proceed this:
> >
> > 1. Extract go/ in apache/arrow to apache/arrow-go like
> > apache/arrow-rs
> > * Filter go/ related commits from apache/arrow and create
> > apache/arrow-go with them like we did for apache/arrow-rs
> > * Remove go/ related codes from apache/arrow
> > 2. Prepare integration test CI like apache/arrow-rs does:
> >
> https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml
> > 3. Prepare release script based on apache/arrow-julia,
> > apache/arrow-adbc and/or apache/arrow-flight-sql-postgresql
>
> I think this is a good idea, but I'm not part of the Go maintainers.
>
> > Cons of this idea:
> >
> > * This is a backward incompatible change
> > * Users need to change their "import" to
> > "github.com/apache/arrow-go/arrow" from
> > "github.com/apache/arrow/go/arrow"
>
> Is there no way to leave some kind of alias or redirection in the
> apache/arrow repository?
>
> Regards
>
> Antoine.
>
Re: [DISCUSS] Split Go release process
Posted by Antoine Pitrou <an...@python.org>.
Hi Kou,
Le 18/07/2024 à 11:33, Sutou Kouhei a écrit :
>
> Here is my idea how to proceed this:
>
> 1. Extract go/ in apache/arrow to apache/arrow-go like
> apache/arrow-rs
> * Filter go/ related commits from apache/arrow and create
> apache/arrow-go with them like we did for apache/arrow-rs
> * Remove go/ related codes from apache/arrow
> 2. Prepare integration test CI like apache/arrow-rs does:
> https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml
> 3. Prepare release script based on apache/arrow-julia,
> apache/arrow-adbc and/or apache/arrow-flight-sql-postgresql
I think this is a good idea, but I'm not part of the Go maintainers.
> Cons of this idea:
>
> * This is a backward incompatible change
> * Users need to change their "import" to
> "github.com/apache/arrow-go/arrow" from
> "github.com/apache/arrow/go/arrow"
Is there no way to leave some kind of alias or redirection in the
apache/arrow repository?
Regards
Antoine.