You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/04/02 11:02:40 UTC
[GitHub] [arrow-datafusion] alamb commented on a change in pull request #2133: Update quarterly roadmap for Q2
alamb commented on a change in pull request #2133:
URL: https://github.com/apache/arrow-datafusion/pull/2133#discussion_r841062815
##########
File path: docs/source/specification/quarterly_roadmap.md
##########
@@ -21,52 +21,65 @@
A quarterly roadmap will be published to give the DataFusion community visibility into the priorities of the projects contributors. This roadmap is not binding.
-## 2022 Q1
+## 2022 Q2
### DataFusion Core
-- Publish official Arrow2 branch
-- Implementation of memory manager (i.e. to enable spilling to disk as needed)
+- IO Improvements
+ - Reading, registering, and writing more file formats from both DataFrame API and SQL
+ - Additional options for IO including partitioning and metadata support
+- Memory Management
+ - Add more operators for memory limited execution
+- Performance
+ - Incorporate row-format into operators such as aggregate
+ - Add row-format benchmarks
+ - Explore LLVM for JIT, with inline Rust functions as the primary goal
+- Documentation
Review comment:
```suggestion
- Improve performance of Sort and Merge using Row Format / JIT expressions
- Documentation
```
##########
File path: docs/source/specification/quarterly_roadmap.md
##########
@@ -21,52 +21,65 @@
A quarterly roadmap will be published to give the DataFusion community visibility into the priorities of the projects contributors. This roadmap is not binding.
-## 2022 Q1
+## 2022 Q2
### DataFusion Core
-- Publish official Arrow2 branch
-- Implementation of memory manager (i.e. to enable spilling to disk as needed)
+- IO Improvements
Review comment:
FYI @tustvold
##########
File path: docs/source/specification/quarterly_roadmap.md
##########
@@ -21,52 +21,65 @@
A quarterly roadmap will be published to give the DataFusion community visibility into the priorities of the projects contributors. This roadmap is not binding.
-## 2022 Q1
+## 2022 Q2
### DataFusion Core
-- Publish official Arrow2 branch
-- Implementation of memory manager (i.e. to enable spilling to disk as needed)
+- IO Improvements
+ - Reading, registering, and writing more file formats from both DataFrame API and SQL
+ - Additional options for IO including partitioning and metadata support
+- Memory Management
+ - Add more operators for memory limited execution
+- Performance
+ - Incorporate row-format into operators such as aggregate
+ - Add row-format benchmarks
+ - Explore LLVM for JIT, with inline Rust functions as the primary goal
+- Documentation
Review comment:
I hope to contribute improvements to the Sort performance (especially for multi-column sorts that include strings) this quarter as well. I don't have any writeup of that yet
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org