You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@arrow.apache.org by GitBox <gi...@apache.org> on 2021/04/27 17:35:21 UTC
[GitHub] [arrow-site] returnString opened a new pull request #107: ARROW-12550: [Website] Write blog post describing new Rust dev processes
returnString opened a new pull request #107:
URL: https://github.com/apache/arrow-site/pull/107
Putting this up in its current state for initial review and feedback from the community.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-site] returnString commented on pull request #107: ARROW-12550: [Website] Write blog post describing new Rust dev processes
Posted by GitBox <gi...@apache.org>.
returnString commented on pull request #107:
URL: https://github.com/apache/arrow-site/pull/107#issuecomment-830616420
Not sure what the policy is on publish dates but if it needs updating to a specific date before merging then please let me know and I can patch that up to today's date; alternatively, maintainer access is enabled for my branch, so PMC members should be able to fix that directly before merging.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-site] andygrove merged pull request #107: ARROW-12550: [Website] Write blog post describing new Rust dev processes
Posted by GitBox <gi...@apache.org>.
andygrove merged pull request #107:
URL: https://github.com/apache/arrow-site/pull/107
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-site] returnString commented on pull request #107: ARROW-12550: [Website] Write blog post describing new Rust dev processes
Posted by GitBox <gi...@apache.org>.
returnString commented on pull request #107:
URL: https://github.com/apache/arrow-site/pull/107#issuecomment-828373193
I'm happy with this initial round of feedback, so promoting this from draft to ready-for-review - thanks all for your effort so far 😄
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-site] andygrove commented on pull request #107: ARROW-12550: [Website] Write blog post describing new Rust dev processes
Posted by GitBox <gi...@apache.org>.
andygrove commented on pull request #107:
URL: https://github.com/apache/arrow-site/pull/107#issuecomment-830656976
LGTM. I will share on the dev@ mailing list as a courtesy as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-site] returnString commented on a change in pull request #107: ARROW-12550: [Website] Write blog post describing new Rust dev processes
Posted by GitBox <gi...@apache.org>.
returnString commented on a change in pull request #107:
URL: https://github.com/apache/arrow-site/pull/107#discussion_r622076355
##########
File path: _posts/2021-04-27-rust-dev-workflow.md
##########
@@ -0,0 +1,79 @@
+---
+layout: post
+title: "A New Development Workflow for Arrow's Rust Implementation"
+date: "2021-04-27 00:00:00"
+author: ruanpa
+categories: [application]
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+The Apache Arrow Rust community is excited to announce that its migration to a new development workflow is now complete! If you're considering Rust as a language for working with columnar data, read on and see how your use case might benefit from our new and improved project setup.
+
+In recent months, members of the community have been working closely with Arrow's [Project Management Committee][1] and other contributors to expand the set of available workflows for Arrow implementations. The goal was to define a new development process that ultimately:
+- Enables a faster release cadence
Review comment:
> I think it looks great. Thank you @returnString !
>
> I left some suggestions that I am not 100% thrilled with -- what I was trying to do was to highlight the specific changes that we hope to make (namely using crate versioning more in line with semver, as well as centralizing on a single tool)
>
> However, it makes the initial bullet points more verbose. I do not feel strongly about this.
>
> All in all, this is really nice
I've taken the spirit of these suggestions and tried to incorporate them without affecting the brevity of the bullet points too significantly - SemVer was worth mentioning for sure!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-site] Dandandan commented on a change in pull request #107: ARROW-12550: [Website] Write blog post describing new Rust dev processes
Posted by GitBox <gi...@apache.org>.
Dandandan commented on a change in pull request #107:
URL: https://github.com/apache/arrow-site/pull/107#discussion_r621575765
##########
File path: _posts/2021-04-27-rust-dev-workflow.md
##########
@@ -0,0 +1,79 @@
+---
+layout: post
+title: "A New Development Workflow for Arrow's Rust Implementation"
+date: "2021-04-27 00:00:00"
+author: ruanpa
+categories: [application]
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+The Apache Arrow Rust community is excited to announce that its migration to a new development workflow is now complete! If you're considering Rust as a language for working with columnar data, read on and see how your use case might benefit from our new and improved project setup.
+
+In recent months, members of the community have been working closely with Arrow's [Project Management Committee][1] and other contributors to expand the set of available workflows for Arrow implementations. The goal was to define a new development process that ultimately:
+- Enables a faster release cadence
+- Encourages maximum participation from the wider community
+- Ensures that we continue to uphold the tenets of [The Apache Way][2]
+
+If you're just here for the highlights, the major outcomes of these discussions are as follows:
+- The Rust projects have moved to separate repositories, outside the main Arrow [monorepo][9]
+ - [arrow-rs][7] for the core Arrow implementation in Rust
+ - [arrow-datafusion][8] for DataFusion and Ballista (more on these projects below!)
+- The Rust community will use GitHub Issues for tracking feature development and issues, replacing the Jira instance maintained by the Apache Software Foundation (ASF)
+
+But why, as a community, have we decided to change our processes? Let's take a slightly more in-depth look at the Rust implementation's needs.
+
+## Project Structure
+The Rust implementation of Arrow actually consists of several distinct projects, or in Rust parlance, ["crates"][3]. In addition to the Arrow crate that contains core functionality such as the in-memory format and inter-process communication, we also maintain:
+- [DataFusion][4]: an extensible in-memory query execution engine using Arrow as its format
+- [Ballista][5]: a distributed compute platform, powered by Apache Arrow and DataFusion
+
+Whilst these projects are all closely related, with many shared contributors, they're very much at different stages in their respective lifecycles. The core Arrow crate, as an implementation of a spec, has strict compatibility requirements with other versions of Arrow, and this is tested via rigorous cross-language integration tests.
+
+However, at the other end of the spectrum, DataFusion and Ballista are still nascent projects in their own right that undergo frequent backwards-incompatible changes. In the old workflow, DataFusion was released in lockstep with Arrow; because DataFusion users often need newly-contributed features or bugfixes on a tighter schedule than Arrow releases, we observed that many people in the community simply resorted to referencing our GitHub repository directly, rather than properly versioned builds on [crates.io][6], Rust's package registry.
+
+Ultimately, the decision was made to split the Rust crates into two separate repositories: [arrow-rs][7] for the core Arrow functionality, and [arrow-datafusion][8] for DataFusion and Ballista. There's still work to be done on determining the exact release workflows for the latter, but this leaves us in a much better position to meet the broader Rust community's expectations of crate versioning and stability.
+
+## Community Participation
+All Apache projects are built on volunteer contribution; it's a core principle of both the ASF and open-source software development more broadly. One point of friction that was observed in the previous workflow for the Rust community in particular was the requirement for issues to be logged in Arrow's Jira project. This step required would-be contributors to first register an account, and then receive a permissions grant to manage tickets.
+
+To streamline this process for new community members, we've taken the decision to migrate to GitHub Issues for tracking both new development work and known bugs that need addressing, and bootstrapped our new repositories by importing their respective tickets from Jira. We hope that this strikes a better balance between organisation and accessibility for prospective contributors.
+
+## Get Involved
+To further improve the onboarding flow for new Arrow contributors, we've begun the process of labelling select issues as "good first issue" in [arrow-rs][11] and [arrow-datafusion][12]. These issues are typically small in scope, but still serve as valuable contributions to the project, and help acclimatise new community members to our development workflows and tools.
Review comment:
```suggestion
To further improve the onboarding flow for new Arrow contributors, we have started the process of labeling select issues as "good first issue" in [arrow-rs][11] and [arrow-datafusion][12]. These issues are small in scope but still serve as valuable contributions to the project. They also help new community members to get familiar with our development workflows and tools.
```
Some stylistic suggestions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-site] returnString commented on a change in pull request #107: ARROW-12550: [Website] Write blog post describing new Rust dev processes
Posted by GitBox <gi...@apache.org>.
returnString commented on a change in pull request #107:
URL: https://github.com/apache/arrow-site/pull/107#discussion_r621505936
##########
File path: _posts/2021-04-27-rust-dev-workflow.md
##########
@@ -0,0 +1,79 @@
+---
+layout: post
+title: "A New Development Workflow for Arrow's Rust Implementation"
+date: "2021-04-27 00:00:00"
+author: ruanpa
+categories: [application]
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+The Apache Arrow Rust community is excited to announce that its migration to a new development workflow is now complete! If you're considering Rust as a language for working with columnar data, read on and see how your use case might benefit from our new and improved project setup.
+
+In recent months, members of the community have been working closely with Arrow's [Project Management Committee][1] and other contributors to expand the set of available workflows for Arrow implementations. The goal was to define a new development process that ultimately:
+- Enables a faster release cadence
+- Encourages maximum participation from the wider community
+- Ensures that we continue to uphold the tenets of [The Apache Way][2]
+
+If you're just here for the highlights, the major outcomes of these discussions are as follows:
+- The Rust projects have moved to separate repositories, outside the main Arrow [monorepo][9]
+ - [arrow-rs][7] for the core Arrow implementation in Rust
+ - [arrow-datafusion][8] for DataFusion and Ballista (more on these projects below!)
+- The Rust community will use GitHub Issues for tracking feature development and issues, replacing the ASF Jira instance
+
+But why, as a community, have we decided to change our processes? Let's take a slightly more in-depth look at the Rust implementation's needs.
+
+## Project Structure
+The Rust implementation of Arrow actually consists of several distinct projects, or in Rust parlance, ["crates"][3]. In addition to the Arrow crate that contains core functionality such as the in-memory format and inter-process communication, we also maintain:
+- [DataFusion][4]: an extensible in-memory query execution engine using Arrow as its format
+- [Ballista][5]: a distributed compute platform built on DataFusion, with Apache Spark compatibility
Review comment:
It seems I imagined this - can't see it in any of the reference material I was looking at yesterday. Will remove!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-site] fitzoh commented on a change in pull request #107: ARROW-12550: [Website] Write blog post describing new Rust dev processes
Posted by GitBox <gi...@apache.org>.
fitzoh commented on a change in pull request #107:
URL: https://github.com/apache/arrow-site/pull/107#discussion_r622178000
##########
File path: _posts/2021-04-27-rust-dev-workflow.md
##########
@@ -0,0 +1,80 @@
+---
+layout: post
+title: "A New Development Workflow for Arrow's Rust Implementation"
+date: "2021-04-27 00:00:00"
+author: ruanpa
+categories: [application]
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+The Apache Arrow Rust community is excited to announce that its migration to a new development workflow is now complete! If you're considering Rust as a language for working with columnar data, read on and see how your use case might benefit from our new and improved project setup.
+
+In recent months, members of the community have been working closely with Arrow's [Project Management Committee][1] and other contributors to expand the set of available workflows for Arrow implementations. The goal was to define a new development process that ultimately:
+- Enables a faster release cadence that adheres to [SemVer][15] where appropriate
+- Encourages maximum participation from the wider community with unified tooling
+- Ensures that we continue to uphold the tenets of [The Apache Way][2]
+
+If you're just here for the highlights, the major outcomes of these discussions are as follows:
+- The Rust projects have moved to separate repositories, outside the main Arrow [monorepo][9]
+ - [arrow-rs][7] for the core Arrow implementation in Rust
+ - [arrow-datafusion][8] for DataFusion and Ballista (more on these projects below!)
+- The Rust community will use GitHub Issues for tracking feature development and issues, replacing the Jira instance maintained by the Apache Software Foundation (ASF)
+
+But why, as a community, have we decided to change our processes? Let's take a slightly more in-depth look at the Rust implementation's needs.
Review comment:
Possibly worth mentioning that this is a shift back towards the process used by Ballista?
Adds the context of "going back to a thing that worked" rather than "let's try something new".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-site] jorgecarleitao commented on a change in pull request #107: ARROW-12550: [Website] Write blog post describing new Rust dev processes
Posted by GitBox <gi...@apache.org>.
jorgecarleitao commented on a change in pull request #107:
URL: https://github.com/apache/arrow-site/pull/107#discussion_r622258786
##########
File path: _posts/2021-04-27-rust-dev-workflow.md
##########
@@ -0,0 +1,80 @@
+---
+layout: post
+title: "A New Development Workflow for Arrow's Rust Implementation"
+date: "2021-04-27 00:00:00"
+author: ruanpa
+categories: [application]
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+The Apache Arrow Rust community is excited to announce that its migration to a new development workflow is now complete! If you're considering Rust as a language for working with columnar data, read on and see how your use case might benefit from our new and improved project setup.
+
+In recent months, members of the community have been working closely with Arrow's [Project Management Committee][1] and other contributors to expand the set of available workflows for Arrow implementations. The goal was to define a new development process that ultimately:
+- Enables a faster release cadence that adheres to [SemVer][15] where appropriate
+- Encourages maximum participation from the wider community with unified tooling
+- Ensures that we continue to uphold the tenets of [The Apache Way][2]
+
+If you're just here for the highlights, the major outcomes of these discussions are as follows:
+- The Rust projects have moved to separate repositories, outside the main Arrow [monorepo][9]
+ - [arrow-rs][7] for the core Arrow implementation in Rust
Review comment:
```suggestion
- [arrow-rs][7] for the Arrow, arrow flight, and parquet implementation in Rust
```
##########
File path: _posts/2021-04-27-rust-dev-workflow.md
##########
@@ -0,0 +1,80 @@
+---
+layout: post
+title: "A New Development Workflow for Arrow's Rust Implementation"
+date: "2021-04-27 00:00:00"
+author: ruanpa
+categories: [application]
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+The Apache Arrow Rust community is excited to announce that its migration to a new development workflow is now complete! If you're considering Rust as a language for working with columnar data, read on and see how your use case might benefit from our new and improved project setup.
+
+In recent months, members of the community have been working closely with Arrow's [Project Management Committee][1] and other contributors to expand the set of available workflows for Arrow implementations. The goal was to define a new development process that ultimately:
+- Enables a faster release cadence that adheres to [SemVer][15] where appropriate
+- Encourages maximum participation from the wider community with unified tooling
+- Ensures that we continue to uphold the tenets of [The Apache Way][2]
+
+If you're just here for the highlights, the major outcomes of these discussions are as follows:
+- The Rust projects have moved to separate repositories, outside the main Arrow [monorepo][9]
Review comment:
I would
```suggestion
- The Rust projects have moved from the [main arrow repository][9] to separate repositories
```
##########
File path: _posts/2021-04-27-rust-dev-workflow.md
##########
@@ -0,0 +1,80 @@
+---
+layout: post
+title: "A New Development Workflow for Arrow's Rust Implementation"
+date: "2021-04-27 00:00:00"
+author: ruanpa
+categories: [application]
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+The Apache Arrow Rust community is excited to announce that its migration to a new development workflow is now complete! If you're considering Rust as a language for working with columnar data, read on and see how your use case might benefit from our new and improved project setup.
+
+In recent months, members of the community have been working closely with Arrow's [Project Management Committee][1] and other contributors to expand the set of available workflows for Arrow implementations. The goal was to define a new development process that ultimately:
+- Enables a faster release cadence that adheres to [SemVer][15] where appropriate
+- Encourages maximum participation from the wider community with unified tooling
+- Ensures that we continue to uphold the tenets of [The Apache Way][2]
+
+If you're just here for the highlights, the major outcomes of these discussions are as follows:
+- The Rust projects have moved to separate repositories, outside the main Arrow [monorepo][9]
+ - [arrow-rs][7] for the core Arrow implementation in Rust
+ - [arrow-datafusion][8] for DataFusion and Ballista (more on these projects below!)
+- The Rust community will use GitHub Issues for tracking feature development and issues, replacing the Jira instance maintained by the Apache Software Foundation (ASF)
+
+But why, as a community, have we decided to change our processes? Let's take a slightly more in-depth look at the Rust implementation's needs.
+
+## Project Structure
+The Rust implementation of Arrow actually consists of several distinct projects, or in Rust parlance, ["crates"][3]. In addition to the Arrow crate that contains core functionality such as the in-memory format and inter-process communication, we also maintain:
+- [DataFusion][4]: an extensible in-memory query execution engine using Arrow as its format
+- [Ballista][5]: a distributed compute platform, powered by Apache Arrow and DataFusion
Review comment:
We have more than 3 crates; within `arrow-rs` we have 4 or 5. This split is more related to the nature of these crates (AFAIK).
##########
File path: _posts/2021-04-27-rust-dev-workflow.md
##########
@@ -0,0 +1,80 @@
+---
+layout: post
+title: "A New Development Workflow for Arrow's Rust Implementation"
+date: "2021-04-27 00:00:00"
+author: ruanpa
+categories: [application]
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+The Apache Arrow Rust community is excited to announce that its migration to a new development workflow is now complete! If you're considering Rust as a language for working with columnar data, read on and see how your use case might benefit from our new and improved project setup.
+
+In recent months, members of the community have been working closely with Arrow's [Project Management Committee][1] and other contributors to expand the set of available workflows for Arrow implementations. The goal was to define a new development process that ultimately:
+- Enables a faster release cadence that adheres to [SemVer][15] where appropriate
+- Encourages maximum participation from the wider community with unified tooling
+- Ensures that we continue to uphold the tenets of [The Apache Way][2]
+
+If you're just here for the highlights, the major outcomes of these discussions are as follows:
+- The Rust projects have moved to separate repositories, outside the main Arrow [monorepo][9]
+ - [arrow-rs][7] for the core Arrow implementation in Rust
+ - [arrow-datafusion][8] for DataFusion and Ballista (more on these projects below!)
+- The Rust community will use GitHub Issues for tracking feature development and issues, replacing the Jira instance maintained by the Apache Software Foundation (ASF)
Review comment:
I would add a third bullet: that the release cycles are independent. IMO that is what enables `SemVer`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-site] emkornfield commented on a change in pull request #107: ARROW-12550: [Website] Write blog post describing new Rust dev processes
Posted by GitBox <gi...@apache.org>.
emkornfield commented on a change in pull request #107:
URL: https://github.com/apache/arrow-site/pull/107#discussion_r624606718
##########
File path: _posts/2021-04-27-rust-dev-workflow.md
##########
@@ -0,0 +1,81 @@
+---
+layout: post
+title: "A New Development Workflow for Arrow's Rust Implementation"
+date: "2021-04-27 00:00:00"
+author: ruanpa
+categories: [application]
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+The Apache Arrow Rust community is excited to announce that its migration to a new development workflow is now complete! If you're considering Rust as a language for working with columnar data, read on and see how your use case might benefit from our new and improved project setup.
+
+In recent months, members of the community have been working closely with Arrow's [Project Management Committee][1] and other contributors to expand the set of available workflows for Arrow implementations. The goal was to define a new development process that ultimately:
+- Enables a faster release cadence that adheres to [SemVer][15] where appropriate
+- Encourages maximum participation from the wider community with unified tooling
+- Ensures that we continue to uphold the tenets of [The Apache Way][2]
+
+If you're just here for the highlights, the major outcomes of these discussions are as follows:
+- The Rust projects have moved to separate repositories, outside the main Arrow [monorepo][9]
+ - [arrow-rs][7] for the core Arrow, Arrow Flight, and Parquet implementations in Rust
+ - [arrow-datafusion][8] for DataFusion and Ballista (more on these projects below!)
+- The Rust community will use GitHub Issues for tracking feature development and issues, replacing the Jira instance maintained by the Apache Software Foundation (ASF)
+- DataFusion and Ballista will follow a new release cycle, independent of the main Arrow releases
+
+But why, as a community, have we decided to change our processes? Let's take a slightly more in-depth look at the Rust implementation's needs.
+
+## Project Structure
+The Rust implementation of Arrow actually consists of several distinct projects, or in Rust parlance, ["crates"][3]. In addition to the core crates, namely `arrow`, `arrow-flight`, and `parquet`, we also maintain:
+- [DataFusion][4]: an extensible in-memory query execution engine using Arrow as its format
+- [Ballista][5]: a distributed compute platform, powered by Apache Arrow and DataFusion
+
+Whilst these projects are all closely related, with many shared contributors, they're very much at different stages in their respective lifecycles. The core Arrow crate, as an implementation of a spec, has strict compatibility requirements with other versions of Arrow, and this is tested via rigorous cross-language integration tests.
+
+However, at the other end of the spectrum, DataFusion and Ballista are still nascent projects in their own right that undergo frequent backwards-incompatible changes. In the old workflow, DataFusion was released in lockstep with Arrow; because DataFusion users often need newly-contributed features or bugfixes on a tighter schedule than Arrow releases, we observed that many people in the community simply resorted to referencing our GitHub repository directly, rather than properly versioned builds on [crates.io][6], Rust's package registry.
+
+Ultimately, the decision was made to split the Rust crates into two separate repositories: [arrow-rs][7] for the core Arrow functionality, and [arrow-datafusion][8] for DataFusion and Ballista. There's still work to be done on determining the exact release workflows for the latter, but this leaves us in a much better position to meet the broader Rust community's expectations of crate versioning and stability.
+
+## Community Participation
+All Apache projects are built on volunteer contribution; it's a core principle of both the ASF and open-source software development more broadly. One point of friction that was observed in the previous workflow for the Rust community in particular was the requirement for issues to be logged in Arrow's Jira project. This step required would-be contributors to first register an account, and then receive a permissions grant to manage tickets.
+
+To streamline this process for new community members, we've taken the decision to migrate to GitHub Issues for tracking both new development work and known bugs that need addressing, and bootstrapped our new repositories by importing their respective tickets from Jira. We hope that this strikes a better balance between organization and accessibility for prospective contributors.
Review comment:
It might help to have language that opening issues before PRs are submitted is recommended to let others know you are working on it and discuss technical issues before work is put into a PR. i.e. this is more a tooling change then anything else.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-site] alamb commented on a change in pull request #107: ARROW-12550: [Website] Write blog post describing new Rust dev processes
Posted by GitBox <gi...@apache.org>.
alamb commented on a change in pull request #107:
URL: https://github.com/apache/arrow-site/pull/107#discussion_r622038640
##########
File path: _posts/2021-04-27-rust-dev-workflow.md
##########
@@ -0,0 +1,79 @@
+---
+layout: post
+title: "A New Development Workflow for Arrow's Rust Implementation"
+date: "2021-04-27 00:00:00"
+author: ruanpa
+categories: [application]
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+The Apache Arrow Rust community is excited to announce that its migration to a new development workflow is now complete! If you're considering Rust as a language for working with columnar data, read on and see how your use case might benefit from our new and improved project setup.
+
+In recent months, members of the community have been working closely with Arrow's [Project Management Committee][1] and other contributors to expand the set of available workflows for Arrow implementations. The goal was to define a new development process that ultimately:
+- Enables a faster release cadence
+- Encourages maximum participation from the wider community
Review comment:
```suggestion
- Encourages maximum participation from the wider community by using only Github (rather than both Github and JIRA)
```
##########
File path: _posts/2021-04-27-rust-dev-workflow.md
##########
@@ -0,0 +1,79 @@
+---
+layout: post
+title: "A New Development Workflow for Arrow's Rust Implementation"
+date: "2021-04-27 00:00:00"
+author: ruanpa
+categories: [application]
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+The Apache Arrow Rust community is excited to announce that its migration to a new development workflow is now complete! If you're considering Rust as a language for working with columnar data, read on and see how your use case might benefit from our new and improved project setup.
+
+In recent months, members of the community have been working closely with Arrow's [Project Management Committee][1] and other contributors to expand the set of available workflows for Arrow implementations. The goal was to define a new development process that ultimately:
+- Enables a faster release cadence
Review comment:
```suggestion
- Enables a faster release cadence and versioning scheme that better matches Cargo's SemVer expectations
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-site] returnString commented on a change in pull request #107: ARROW-12550: [Website] Write blog post describing new Rust dev processes
Posted by GitBox <gi...@apache.org>.
returnString commented on a change in pull request #107:
URL: https://github.com/apache/arrow-site/pull/107#discussion_r625630839
##########
File path: _posts/2021-04-27-rust-dev-workflow.md
##########
@@ -0,0 +1,81 @@
+---
+layout: post
+title: "A New Development Workflow for Arrow's Rust Implementation"
+date: "2021-04-27 00:00:00"
+author: ruanpa
+categories: [application]
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+The Apache Arrow Rust community is excited to announce that its migration to a new development workflow is now complete! If you're considering Rust as a language for working with columnar data, read on and see how your use case might benefit from our new and improved project setup.
+
+In recent months, members of the community have been working closely with Arrow's [Project Management Committee][1] and other contributors to expand the set of available workflows for Arrow implementations. The goal was to define a new development process that ultimately:
+- Enables a faster release cadence that adheres to [SemVer][15] where appropriate
+- Encourages maximum participation from the wider community with unified tooling
+- Ensures that we continue to uphold the tenets of [The Apache Way][2]
+
+If you're just here for the highlights, the major outcomes of these discussions are as follows:
+- The Rust projects have moved to separate repositories, outside the main Arrow [monorepo][9]
+ - [arrow-rs][7] for the core Arrow, Arrow Flight, and Parquet implementations in Rust
+ - [arrow-datafusion][8] for DataFusion and Ballista (more on these projects below!)
+- The Rust community will use GitHub Issues for tracking feature development and issues, replacing the Jira instance maintained by the Apache Software Foundation (ASF)
+- DataFusion and Ballista will follow a new release cycle, independent of the main Arrow releases
+
+But why, as a community, have we decided to change our processes? Let's take a slightly more in-depth look at the Rust implementation's needs.
+
+## Project Structure
+The Rust implementation of Arrow actually consists of several distinct projects, or in Rust parlance, ["crates"][3]. In addition to the core crates, namely `arrow`, `arrow-flight`, and `parquet`, we also maintain:
+- [DataFusion][4]: an extensible in-memory query execution engine using Arrow as its format
+- [Ballista][5]: a distributed compute platform, powered by Apache Arrow and DataFusion
+
+Whilst these projects are all closely related, with many shared contributors, they're very much at different stages in their respective lifecycles. The core Arrow crate, as an implementation of a spec, has strict compatibility requirements with other versions of Arrow, and this is tested via rigorous cross-language integration tests.
+
+However, at the other end of the spectrum, DataFusion and Ballista are still nascent projects in their own right that undergo frequent backwards-incompatible changes. In the old workflow, DataFusion was released in lockstep with Arrow; because DataFusion users often need newly-contributed features or bugfixes on a tighter schedule than Arrow releases, we observed that many people in the community simply resorted to referencing our GitHub repository directly, rather than properly versioned builds on [crates.io][6], Rust's package registry.
+
+Ultimately, the decision was made to split the Rust crates into two separate repositories: [arrow-rs][7] for the core Arrow functionality, and [arrow-datafusion][8] for DataFusion and Ballista. There's still work to be done on determining the exact release workflows for the latter, but this leaves us in a much better position to meet the broader Rust community's expectations of crate versioning and stability.
+
+## Community Participation
+All Apache projects are built on volunteer contribution; it's a core principle of both the ASF and open-source software development more broadly. One point of friction that was observed in the previous workflow for the Rust community in particular was the requirement for issues to be logged in Arrow's Jira project. This step required would-be contributors to first register an account, and then receive a permissions grant to manage tickets.
+
+To streamline this process for new community members, we've taken the decision to migrate to GitHub Issues for tracking both new development work and known bugs that need addressing, and bootstrapped our new repositories by importing their respective tickets from Jira. We hope that this strikes a better balance between organization and accessibility for prospective contributors.
Review comment:
Thanks for the feedback @emkornfield! I've incorporated it into that section.
@andygrove I think we're good to go 😄
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-site] alamb commented on pull request #107: ARROW-12550: [Website] Write blog post describing new Rust dev processes
Posted by GitBox <gi...@apache.org>.
alamb commented on pull request #107:
URL: https://github.com/apache/arrow-site/pull/107#issuecomment-830610780
@andygrove is this one good to go? I think it is ready
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-site] returnString commented on a change in pull request #107: ARROW-12550: [Website] Write blog post describing new Rust dev processes
Posted by GitBox <gi...@apache.org>.
returnString commented on a change in pull request #107:
URL: https://github.com/apache/arrow-site/pull/107#discussion_r622359077
##########
File path: _posts/2021-04-27-rust-dev-workflow.md
##########
@@ -0,0 +1,80 @@
+---
+layout: post
+title: "A New Development Workflow for Arrow's Rust Implementation"
+date: "2021-04-27 00:00:00"
+author: ruanpa
+categories: [application]
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+The Apache Arrow Rust community is excited to announce that its migration to a new development workflow is now complete! If you're considering Rust as a language for working with columnar data, read on and see how your use case might benefit from our new and improved project setup.
+
+In recent months, members of the community have been working closely with Arrow's [Project Management Committee][1] and other contributors to expand the set of available workflows for Arrow implementations. The goal was to define a new development process that ultimately:
+- Enables a faster release cadence that adheres to [SemVer][15] where appropriate
+- Encourages maximum participation from the wider community with unified tooling
+- Ensures that we continue to uphold the tenets of [The Apache Way][2]
+
+If you're just here for the highlights, the major outcomes of these discussions are as follows:
+- The Rust projects have moved to separate repositories, outside the main Arrow [monorepo][9]
Review comment:
I was trying to avoid repeating the word repository too often - difficult in a post on dev and release workflows, admittedly 😅
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-site] github-actions[bot] commented on pull request #107: ARROW-12550: [Website] Write blog post describing new Rust dev processes
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #107:
URL: https://github.com/apache/arrow-site/pull/107#issuecomment-827786048
https://issues.apache.org/jira/browse/ARROW-12550
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-site] fitzoh commented on a change in pull request #107: ARROW-12550: [Website] Write blog post describing new Rust dev processes
Posted by GitBox <gi...@apache.org>.
fitzoh commented on a change in pull request #107:
URL: https://github.com/apache/arrow-site/pull/107#discussion_r621501817
##########
File path: _posts/2021-04-27-rust-dev-workflow.md
##########
@@ -0,0 +1,79 @@
+---
+layout: post
+title: "A New Development Workflow for Arrow's Rust Implementation"
+date: "2021-04-27 00:00:00"
+author: ruanpa
+categories: [application]
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+The Apache Arrow Rust community is excited to announce that its migration to a new development workflow is now complete! If you're considering Rust as a language for working with columnar data, read on and see how your use case might benefit from our new and improved project setup.
+
+In recent months, members of the community have been working closely with Arrow's [Project Management Committee][1] and other contributors to expand the set of available workflows for Arrow implementations. The goal was to define a new development process that ultimately:
+- Enables a faster release cadence
+- Encourages maximum participation from the wider community
+- Ensures that we continue to uphold the tenets of [The Apache Way][2]
+
+If you're just here for the highlights, the major outcomes of these discussions are as follows:
+- The Rust projects have moved to separate repositories, outside the main Arrow [monorepo][9]
+ - [arrow-rs][7] for the core Arrow implementation in Rust
+ - [arrow-datafusion][8] for DataFusion and Ballista (more on these projects below!)
+- The Rust community will use GitHub Issues for tracking feature development and issues, replacing the ASF Jira instance
Review comment:
`ASF` is used a couple times, worth defining?
##########
File path: _posts/2021-04-27-rust-dev-workflow.md
##########
@@ -0,0 +1,79 @@
+---
+layout: post
+title: "A New Development Workflow for Arrow's Rust Implementation"
+date: "2021-04-27 00:00:00"
+author: ruanpa
+categories: [application]
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+The Apache Arrow Rust community is excited to announce that its migration to a new development workflow is now complete! If you're considering Rust as a language for working with columnar data, read on and see how your use case might benefit from our new and improved project setup.
+
+In recent months, members of the community have been working closely with Arrow's [Project Management Committee][1] and other contributors to expand the set of available workflows for Arrow implementations. The goal was to define a new development process that ultimately:
+- Enables a faster release cadence
+- Encourages maximum participation from the wider community
+- Ensures that we continue to uphold the tenets of [The Apache Way][2]
+
+If you're just here for the highlights, the major outcomes of these discussions are as follows:
+- The Rust projects have moved to separate repositories, outside the main Arrow [monorepo][9]
+ - [arrow-rs][7] for the core Arrow implementation in Rust
+ - [arrow-datafusion][8] for DataFusion and Ballista (more on these projects below!)
+- The Rust community will use GitHub Issues for tracking feature development and issues, replacing the ASF Jira instance
+
+But why, as a community, have we decided to change our processes? Let's take a slightly more in-depth look at the Rust implementation's needs.
+
+## Project Structure
+The Rust implementation of Arrow actually consists of several distinct projects, or in Rust parlance, ["crates"][3]. In addition to the Arrow crate that contains core functionality such as the in-memory format and inter-process communication, we also maintain:
+- [DataFusion][4]: an extensible in-memory query execution engine using Arrow as its format
+- [Ballista][5]: a distributed compute platform built on DataFusion, with Apache Spark compatibility
Review comment:
I was unaware of Spark compatibility and couldn't find that claim on a very cursory search.
Is it compatible with Spark or comparable to Spark?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-site] returnString commented on a change in pull request #107: ARROW-12550: [Website] Write blog post describing new Rust dev processes
Posted by GitBox <gi...@apache.org>.
returnString commented on a change in pull request #107:
URL: https://github.com/apache/arrow-site/pull/107#discussion_r621590257
##########
File path: _posts/2021-04-27-rust-dev-workflow.md
##########
@@ -0,0 +1,79 @@
+---
+layout: post
+title: "A New Development Workflow for Arrow's Rust Implementation"
+date: "2021-04-27 00:00:00"
+author: ruanpa
+categories: [application]
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+The Apache Arrow Rust community is excited to announce that its migration to a new development workflow is now complete! If you're considering Rust as a language for working with columnar data, read on and see how your use case might benefit from our new and improved project setup.
+
+In recent months, members of the community have been working closely with Arrow's [Project Management Committee][1] and other contributors to expand the set of available workflows for Arrow implementations. The goal was to define a new development process that ultimately:
+- Enables a faster release cadence
+- Encourages maximum participation from the wider community
+- Ensures that we continue to uphold the tenets of [The Apache Way][2]
+
+If you're just here for the highlights, the major outcomes of these discussions are as follows:
+- The Rust projects have moved to separate repositories, outside the main Arrow [monorepo][9]
+ - [arrow-rs][7] for the core Arrow implementation in Rust
+ - [arrow-datafusion][8] for DataFusion and Ballista (more on these projects below!)
+- The Rust community will use GitHub Issues for tracking feature development and issues, replacing the Jira instance maintained by the Apache Software Foundation (ASF)
+
+But why, as a community, have we decided to change our processes? Let's take a slightly more in-depth look at the Rust implementation's needs.
+
+## Project Structure
+The Rust implementation of Arrow actually consists of several distinct projects, or in Rust parlance, ["crates"][3]. In addition to the Arrow crate that contains core functionality such as the in-memory format and inter-process communication, we also maintain:
+- [DataFusion][4]: an extensible in-memory query execution engine using Arrow as its format
+- [Ballista][5]: a distributed compute platform, powered by Apache Arrow and DataFusion
+
+Whilst these projects are all closely related, with many shared contributors, they're very much at different stages in their respective lifecycles. The core Arrow crate, as an implementation of a spec, has strict compatibility requirements with other versions of Arrow, and this is tested via rigorous cross-language integration tests.
+
+However, at the other end of the spectrum, DataFusion and Ballista are still nascent projects in their own right that undergo frequent backwards-incompatible changes. In the old workflow, DataFusion was released in lockstep with Arrow; because DataFusion users often need newly-contributed features or bugfixes on a tighter schedule than Arrow releases, we observed that many people in the community simply resorted to referencing our GitHub repository directly, rather than properly versioned builds on [crates.io][6], Rust's package registry.
+
+Ultimately, the decision was made to split the Rust crates into two separate repositories: [arrow-rs][7] for the core Arrow functionality, and [arrow-datafusion][8] for DataFusion and Ballista. There's still work to be done on determining the exact release workflows for the latter, but this leaves us in a much better position to meet the broader Rust community's expectations of crate versioning and stability.
+
+## Community Participation
+All Apache projects are built on volunteer contribution; it's a core principle of both the ASF and open-source software development more broadly. One point of friction that was observed in the previous workflow for the Rust community in particular was the requirement for issues to be logged in Arrow's Jira project. This step required would-be contributors to first register an account, and then receive a permissions grant to manage tickets.
+
+To streamline this process for new community members, we've taken the decision to migrate to GitHub Issues for tracking both new development work and known bugs that need addressing, and bootstrapped our new repositories by importing their respective tickets from Jira. We hope that this strikes a better balance between organisation and accessibility for prospective contributors.
+
+## Get Involved
+To further improve the onboarding flow for new Arrow contributors, we've begun the process of labelling select issues as "good first issue" in [arrow-rs][11] and [arrow-datafusion][12]. These issues are typically small in scope, but still serve as valuable contributions to the project, and help acclimatise new community members to our development workflows and tools.
Review comment:
Ah, the old single vs double L, a classic in English dialect wars! Lemme scan through the blog post history, I'll standardise (heh, I'm picking a side already with that S) on either British or American English depending on which is more common. There's probably other bits that need unifying if so.
I'll try and incorporate these suggestions in some fashion, clarity is always a good goal 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-site] andygrove commented on a change in pull request #107: ARROW-12550: [Website] Write blog post describing new Rust dev processes
Posted by GitBox <gi...@apache.org>.
andygrove commented on a change in pull request #107:
URL: https://github.com/apache/arrow-site/pull/107#discussion_r625194249
##########
File path: _posts/2021-04-27-rust-dev-workflow.md
##########
@@ -0,0 +1,81 @@
+---
+layout: post
+title: "A New Development Workflow for Arrow's Rust Implementation"
+date: "2021-04-27 00:00:00"
+author: ruanpa
+categories: [application]
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+The Apache Arrow Rust community is excited to announce that its migration to a new development workflow is now complete! If you're considering Rust as a language for working with columnar data, read on and see how your use case might benefit from our new and improved project setup.
+
+In recent months, members of the community have been working closely with Arrow's [Project Management Committee][1] and other contributors to expand the set of available workflows for Arrow implementations. The goal was to define a new development process that ultimately:
+- Enables a faster release cadence that adheres to [SemVer][15] where appropriate
+- Encourages maximum participation from the wider community with unified tooling
+- Ensures that we continue to uphold the tenets of [The Apache Way][2]
+
+If you're just here for the highlights, the major outcomes of these discussions are as follows:
+- The Rust projects have moved to separate repositories, outside the main Arrow [monorepo][9]
+ - [arrow-rs][7] for the core Arrow, Arrow Flight, and Parquet implementations in Rust
+ - [arrow-datafusion][8] for DataFusion and Ballista (more on these projects below!)
+- The Rust community will use GitHub Issues for tracking feature development and issues, replacing the Jira instance maintained by the Apache Software Foundation (ASF)
+- DataFusion and Ballista will follow a new release cycle, independent of the main Arrow releases
+
+But why, as a community, have we decided to change our processes? Let's take a slightly more in-depth look at the Rust implementation's needs.
+
+## Project Structure
+The Rust implementation of Arrow actually consists of several distinct projects, or in Rust parlance, ["crates"][3]. In addition to the core crates, namely `arrow`, `arrow-flight`, and `parquet`, we also maintain:
+- [DataFusion][4]: an extensible in-memory query execution engine using Arrow as its format
+- [Ballista][5]: a distributed compute platform, powered by Apache Arrow and DataFusion
+
+Whilst these projects are all closely related, with many shared contributors, they're very much at different stages in their respective lifecycles. The core Arrow crate, as an implementation of a spec, has strict compatibility requirements with other versions of Arrow, and this is tested via rigorous cross-language integration tests.
+
+However, at the other end of the spectrum, DataFusion and Ballista are still nascent projects in their own right that undergo frequent backwards-incompatible changes. In the old workflow, DataFusion was released in lockstep with Arrow; because DataFusion users often need newly-contributed features or bugfixes on a tighter schedule than Arrow releases, we observed that many people in the community simply resorted to referencing our GitHub repository directly, rather than properly versioned builds on [crates.io][6], Rust's package registry.
+
+Ultimately, the decision was made to split the Rust crates into two separate repositories: [arrow-rs][7] for the core Arrow functionality, and [arrow-datafusion][8] for DataFusion and Ballista. There's still work to be done on determining the exact release workflows for the latter, but this leaves us in a much better position to meet the broader Rust community's expectations of crate versioning and stability.
+
+## Community Participation
+All Apache projects are built on volunteer contribution; it's a core principle of both the ASF and open-source software development more broadly. One point of friction that was observed in the previous workflow for the Rust community in particular was the requirement for issues to be logged in Arrow's Jira project. This step required would-be contributors to first register an account, and then receive a permissions grant to manage tickets.
+
+To streamline this process for new community members, we've taken the decision to migrate to GitHub Issues for tracking both new development work and known bugs that need addressing, and bootstrapped our new repositories by importing their respective tickets from Jira. We hope that this strikes a better balance between organization and accessibility for prospective contributors.
Review comment:
@returnString If you could address this point and update the publish date I think we can go ahead and get this published.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-site] alamb commented on pull request #107: ARROW-12550: [Website] Write blog post describing new Rust dev processes
Posted by GitBox <gi...@apache.org>.
alamb commented on pull request #107:
URL: https://github.com/apache/arrow-site/pull/107#issuecomment-832602344
🎉 Thanks again @returnString everyone else who contributed to this post
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org