You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "GlutenPerfBot (via GitHub)" <gi...@apache.org> on 2025/10/10 20:16:48 UTC
[D] October 10, 2025: Weekly Status Update in Gluten [incubator-gluten]
GitHub user GlutenPerfBot created a discussion: October 10, 2025: Weekly Status Update in Gluten
*This weekly update is generated by LLMs. You're welcome to join our [Github](https://github.com/apache/incubator-gluten/discussions) for in-depth discussions.*
## Overall Activity Summary
The past 7 days have been dominated by release-prep fever: 1.5.0 is being finalized, so version-bump PRs, doc polish and back-ports are everywhere. Meanwhile Velox keeps moving (daily bumps), Iceberg & Delta lake-house features are expanding, and a long tail of micro-optimizations (memcpy, vector copy, hash-table configs) show the community is squeezing every last cycle out of the engine.
## Key Ongoing Projects
- **1.5.0 release** – @PHILO-HE drove #10855 (copyright NOTICE) and #10858 (doc version fixes); @zhztheplayer updated Flink docs for RC1 (#10861)
- **Native Delta/Iceberg write** – @zhztheplayer’s 1 787-line PoC (#10801) plus companion fixes (#10822, #10830) bring off-loaded Parquet writer for Delta 3.3 on Spark 3.5; @Zouxxyy landed dynamic-partition overwrite for Iceberg (#10823, #10760)
- **Velox daily sync** – @GlutenPerfBot keeps us current with upstream (#10862, #10854, #10849, #10847, #10844)
- **Performance micro-optimizations** – @zhouyuan and @zhli1142015 attack redundant memcpy in Column-to-Row (#10824, #10825); @zml1206 adds months_between function support (#10782)
## Priority Items
- **Performance regression** – #10811 by @beliefer flips propagateIgnoreNullKeys default after TPC-DS q87-90 slowdown; needs review
- **Spill level bug** – #10845 reports HashBuild operator spill level exceeding maximum; @himanshu-goyal-hxh needs help
- **OOM in UnboundedWindowFunctionFrame** – #10500 by @zml1206 affects entire-partition window queries; needs memory management fix
- **Build stability** – #10804 (Iceberg jar version clash on ARM) closed but similar flakes in #10756 (CH AQE test) still open; @jinchengchenghh and @lgbo-ustc are chasing
## Notable Discussions
- #10841: Weekly status bot summary shows sustained velocity across Velox/CH/Flink with 38 merged PRs, 29 open
- #4392: Stage level resource management discussion for offheap/onheap memory conflict handling
- #8782: OAP/Velox upstreaming tracking issue with multiple blocked PRs
## Emerging Trends
- **Lake-house parity race** – Iceberg & Delta getting simultaneous native read/write features; expect similar push for Hudi
- **Micro-perf everywhere** – C2R memory zeroing, vector flatten avoidance, hash-table tuning: death-by-a-thousand-cuts phase
- **Multi-backend convergence** – same feature (input-file expr, cudf connector, config API) implemented for both Velox and CH within days
- **Release automation** – new GHA workflows and packaging fixes show infra maturing
## Good First Issues
- #6814: Implement MakeYMInterval for ClickHouse – simple date-interval UDF, great to learn CH function registry
- #4730: Add date_from_unix_date – follows existing date-function pattern
- #6807: Support split_part string function – self-contained, touches parser + UDF stub
- #6812: Expose SparkPartitionID – reuses existing partition metadata, no native code needed
- #6815: Add MapZipWith – entry-level map function; good excuse to peek into Velox/CH map kernels
All need basic C++ or Scala, come with clear signatures, and have prior examples to copy.
GitHub link: https://github.com/apache/incubator-gluten/discussions/10864
----
This is an automatically sent email for commits@gluten.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@gluten.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org