You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/03/25 16:59:47 UTC
[GitHub] [arrow-datafusion] alamb opened a new issue #2095: Create next DataFusion release (after 7.0)
alamb opened a new issue #2095:
URL: https://github.com/apache/arrow-datafusion/issues/2095
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
We released datafusion 7.0.0 about a month ago https://crates.io/crates/datafusion/7.0.0
We should figure out when to release the next one
**Describe the solution you'd like**
Plan out the next release(s) of DataFusion. Also figure out if we want to do a maintenance release (e.g. 7.0.1 / 7.1.0) or a release from master (8.0.0).
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Brought up by @silence-coding here: https://github.com/apache/arrow-datafusion/pull/2066#issuecomment-1078715516
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on issue #2095: Create next DataFusion release (after 7.0)
Posted by GitBox <gi...@apache.org>.
alamb commented on issue #2095:
URL: https://github.com/apache/arrow-datafusion/issues/2095#issuecomment-1079900357
> Agree to make 3-layer releases: major, minor, bug fix.
Arrow C++ does major quarterly releases; I have not seen a minor release (e.g. 6.1.0) in the last year. Occasionally there are patch releases but it is infrequent and typically once per major release.
I agree the three release sounds ideal as well.
> minor release (from master), every 1 month
If we intended to conform to ["semantic versioning"](https://doc.rust-lang.org/cargo/reference/semver.html) in the rust style, it is a challenge to release minor versions from `master`. For the minor (e.g. 7.0.0 to 7.1.0) release to be semantically versioned no breaking API changes can be introduced which would restrict what we can put on master
> Another question is whether it is necessary to maintain the same version for different modules, like Ballista, datafusion-data-access (newly splitted one).
I do not think it is necessary to keep the same versions. I keep the versions of arrow-rs/arrow-flight/parquet in sync because it lowers the release overhead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] silence-coding commented on issue #2095: Create next DataFusion release (after 7.0)
Posted by GitBox <gi...@apache.org>.
silence-coding commented on issue #2095:
URL: https://github.com/apache/arrow-datafusion/issues/2095#issuecomment-1079558000
I think datafusion can release a small version (e.g. 7.0.1) once a month and a large version (e.g. 8.0.0-alpha) when there are major changes. The reason is that the pull request of DataFusion is frequent. Therefore, a stable release plan is required.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] houqp commented on issue #2095: Create next DataFusion release (after 7.0)
Posted by GitBox <gi...@apache.org>.
houqp commented on issue #2095:
URL: https://github.com/apache/arrow-datafusion/issues/2095#issuecomment-1080016194
Agree that backporting patches to a stable branch is a very time consuming work so better not commit to it until we see strong need from our users or we have a maintainer who can allocate dedicated time to maintain the stable branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] jychen7 edited a comment on issue #2095: Create next DataFusion release (after 7.0)
Posted by GitBox <gi...@apache.org>.
jychen7 edited a comment on issue #2095:
URL: https://github.com/apache/arrow-datafusion/issues/2095#issuecomment-1079785765
[Arrow C++ (official)](https://arrow.apache.org/release/) seems to have major release quarterly and as of 2022-02, it is 7.0.0.
As of 2022-03, [Arrow Rust](https://github.com/apache/arrow-rs/tags) have reach 11.0.0 and [Arrow Datafusion](https://github.com/apache/arrow-datafusion/tags) is 7.0.0.
I think Datafusion can have similar release plan as Arrow (C++)
- major release (from master), every 3 month
- minor release (from master), every 1 month
- patch release (from previous minor release branch) for bugs (review whether need release every week)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] jychen7 edited a comment on issue #2095: Create next DataFusion release (after 7.0)
Posted by GitBox <gi...@apache.org>.
jychen7 edited a comment on issue #2095:
URL: https://github.com/apache/arrow-datafusion/issues/2095#issuecomment-1079785765
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] HaoYang670 commented on issue #2095: Create next DataFusion release (after 7.0)
Posted by GitBox <gi...@apache.org>.
HaoYang670 commented on issue #2095:
URL: https://github.com/apache/arrow-datafusion/issues/2095#issuecomment-1079686587
Maybe we could imitate the style of Apache Spark.
1. Major release (such as 8.0.0)(irregularly, but may be 2~3 years): significant new features, optimizations in architecture, or the back compatibility is broken.
2. Minor release (such as 7.1, 7.2 ...)(around half a year): Performance improvement and small features
3. Maintenance release (such as 7.0.1, 7.0.2)(regularly released monthly) : focus on bugs and stability. New features should not be introduced.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] jychen7 edited a comment on issue #2095: Create next DataFusion release (after 7.0)
Posted by GitBox <gi...@apache.org>.
jychen7 edited a comment on issue #2095:
URL: https://github.com/apache/arrow-datafusion/issues/2095#issuecomment-1079785765
[Arrow (non-rust)](https://arrow.apache.org/release/) seems to have major release quarterly and as of 2022-02, it is 7.0.0. As of 2022-03, [Arrow Rust](https://github.com/apache/arrow-rs/tags) have reach 11.0.0 and [Arrow Datafusion](https://github.com/apache/arrow-datafusion/tags) is 7.0.0.
I think Datafusion can have similar release plan as Arrow (non-rust)
- major release (from master), every 3 month
- minor release (from master), every 1 month
- patch release (from previous minor release branch) for bugs (review whether need release every week)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] yahoNanJing commented on issue #2095: Create next DataFusion release (after 7.0)
Posted by GitBox <gi...@apache.org>.
yahoNanJing commented on issue #2095:
URL: https://github.com/apache/arrow-datafusion/issues/2095#issuecomment-1079807606
Agree to make 3-layer releases: major, minor, bug fix.
Another question is whether it is necessary to maintain the same version for different modules, like Ballista, datafusion-data-access (newly splitted one).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on issue #2095: Create next DataFusion release (after 7.0)
Posted by GitBox <gi...@apache.org>.
alamb commented on issue #2095:
URL: https://github.com/apache/arrow-datafusion/issues/2095#issuecomment-1082381567
> @alamb if I understand correctly, our next major release wil be around 2022-05-14 (2nd weekend of May). And next possible minor release will be around 2022-04-09 (2nd weekend of Apr).
I think that would be reasonable
> we ask contributor who want minor/patch release to create PR to maint-* branch after original PR is merged to master? I draft the doc update at https://github.com/apache/arrow-datafusion/pull/2110
Yes, thank you
> As volunteer, I would help to
❤️ thank you so much!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] silence-coding commented on issue #2095: Create next DataFusion release (after 7.0)
Posted by GitBox <gi...@apache.org>.
silence-coding commented on issue #2095:
URL: https://github.com/apache/arrow-datafusion/issues/2095#issuecomment-1079654699
I agree with you very much. I suggest that you can post a bulletin in ReadMe to recruit volunteers to help manage the publishing. It may not be obvious to track the issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on issue #2095: Create next DataFusion release (after 7.0)
Posted by GitBox <gi...@apache.org>.
alamb commented on issue #2095:
URL: https://github.com/apache/arrow-datafusion/issues/2095#issuecomment-1081062492
Ok, since @jychen7 has volunteered, let's give it a try for a release or two of `datafusion 7.x`
I have created a 7.x maintenance branch
* https://github.com/apache/arrow-datafusion/tree/maint-7.x
The next steps would be to decide on some content to backport (via cherry-pick) that are semantically compatible.
To do so I suggest:
1. Create a new PR for each change you would like to release in the 7.x line against the `maint-7.x` branch
2. Tag me on the PR -- I'll review and merge
3. When we are ready to release this next version, we can update the release notes / changelog and I'll propose an official release.
Sound good @jychen7 ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] jychen7 commented on issue #2095: Create next DataFusion release (after 7.0)
Posted by GitBox <gi...@apache.org>.
jychen7 commented on issue #2095:
URL: https://github.com/apache/arrow-datafusion/issues/2095#issuecomment-1081329134
@alamb if I understand correctly, our next major release wil be around 2022-05-14 (2nd weekend of May). And next possible minor release will be around 2022-04-09 (2nd weekend of Apr).
we ask contributor who want minor/patch release to create PR to `maint-*` branch after original PR is merged to master? I draft the doc update at https://github.com/apache/arrow-datafusion/pull/2110
As volunteer, I would help to
1. every week, check how many PRs to maintainance branch (e.g. search by `base:maint-7.x`). If any,
- confirm commit is cherry-pick from `master`
- confirms is minor (non API breaking) change
- tag you for review
2. draft changelog for release every month if major/minor release need
ps: 1 may be automate in Github workflow in future if need
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on issue #2095: Create next DataFusion release (after 7.0)
Posted by GitBox <gi...@apache.org>.
alamb commented on issue #2095:
URL: https://github.com/apache/arrow-datafusion/issues/2095#issuecomment-1079900657
The challenge I predict we will encounter is getting the time to manage the releases (aka reviewing PRs, decide what to backport, backporting, making release notes and version bumps).
I don't think the work is "hard" per se but it does take sustained time and effort
Maybe we could start with
* major release (from master), every 3 month
* minor/patch release (from previous minor release branch) for bugs and minor features (released on on demand / every month)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] jychen7 commented on issue #2095: Create next DataFusion release (after 7.0)
Posted by GitBox <gi...@apache.org>.
jychen7 commented on issue #2095:
URL: https://github.com/apache/arrow-datafusion/issues/2095#issuecomment-1079928346
> Does anyone want to volunteer to manage such release(s)?
I would love to
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] jychen7 commented on issue #2095: Create next DataFusion release (after 7.0)
Posted by GitBox <gi...@apache.org>.
jychen7 commented on issue #2095:
URL: https://github.com/apache/arrow-datafusion/issues/2095#issuecomment-1079785765
[Arrow (non-rust)](https://arrow.apache.org/release/) seems to have major release quarterly and as of 2022-02, it is 7.0.0. As of 2022-03, Arrow Rust have reach 11.0.0 and Arrow Datafusion is 7.0.0.
I think Datafusion can have similar release plan as Arrow (non-rust)
- major release (from master), every 3 month
- minor release (from master), every 1 month
- patch release (from previous minor release branch) for bugs (review whether need release every week)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on issue #2095: Create next DataFusion release (after 7.0)
Posted by GitBox <gi...@apache.org>.
alamb commented on issue #2095:
URL: https://github.com/apache/arrow-datafusion/issues/2095#issuecomment-1079900713
Does anyone want to volunteer to manage such release(s)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on issue #2095: Create next DataFusion release (after 7.0)
Posted by GitBox <gi...@apache.org>.
alamb commented on issue #2095:
URL: https://github.com/apache/arrow-datafusion/issues/2095#issuecomment-1079648483
I would be happy to help support more incremental releases of datafusion, but I probably don't have time to manage the whole thing
What I think would be needed is:
1. A stable branch (`release_7.x`) -- which I can make
2. people to make PRs that cherry-pick changes from `master` to that stable branch
3. A regular release from the release branch
I am happy to do the mechanics of creating a branch and release artifacts, but I would need help from the community backporting / cherry-picking backwards compatible changes to it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org