You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@auron.apache.org by "weimingdiit (via GitHub)" <gi...@apache.org> on 2026/07/25 03:02:18 UTC
[PR] [AURON #2425] Support Iceberg STARTS_WITH pruning in native scan [auron]
weimingdiit opened a new pull request, #2426:
URL: https://github.com/apache/auron/pull/2426
**Which issue does this PR close?**
Closes #2425
**Rationale for this change**
Auron Iceberg native scan already collects Iceberg residual filters and converts supported filters into native scan pruning predicates.
Iceberg can produce `STARTS_WITH` filters for prefix string predicates, such as `LIKE 'a%'`, but Auron currently does not convert this Iceberg operation. As a result, the query can still use a native post-scan filter, but the starts-with predicate is not used for native scan pruning.
**What changes are included in this PR?**
Adds Iceberg `STARTS_WITH` conversion for string columns in `IcebergScanSupport`.
Reuses the existing Spark `StartsWith` native pruning conversion.
Keeps other string and binary pruning behavior unchanged.
Adds an Iceberg integration test for prefix LIKE pruning.
**Are there any user-facing changes?**
No user-facing API changes. More Iceberg prefix string filters can now be passed into native scan pruning.
**How was this patch tested?**
UT.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@auron.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AURON #2425] Support Iceberg STARTS_WITH pruning in native scan [auron]
Posted by "slfan1989 (via GitHub)" <gi...@apache.org>.
slfan1989 merged PR #2426:
URL: https://github.com/apache/auron/pull/2426
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@auron.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AURON #2425] Support Iceberg STARTS_WITH pruning in native scan [auron]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #2426:
URL: https://github.com/apache/auron/pull/2426#discussion_r3649507491
##########
thirdparty/auron-iceberg/src/test/scala/org/apache/auron/iceberg/AuronIcebergIntegrationSuite.scala:
##########
@@ -466,6 +466,31 @@ class AuronIcebergIntegrationSuite
}
}
+ test("iceberg scan pushes STARTS_WITH filters into native scan pruning predicates") {
+ withTable("local.db.t_residual_starts_with") {
+ sql("create table local.db.t_residual_starts_with (id int, v string) using iceberg")
+ sql("""
+ |insert into local.db.t_residual_starts_with
+ |values (1, 'alpha'), (2, 'beta'), (3, 'atom'), (4, null)
+ |""".stripMargin)
+ val df = sql("""
+ |select * from local.db.t_residual_starts_with
+ |where v like 'a%'
+ |""".stripMargin)
+ checkAnswer(df, Seq(Row(1, "alpha"), Row(3, "atom")))
+ val nativeScanPlan = icebergScanPlan(df)
+ assert(nativeScanPlan.nonEmpty)
+ val pruningPredicateText = nativeScanPlan.get.pruningPredicates.mkString("\n")
+ assert(
+ pruningPredicateText.contains("name: \"starts_with\"") &&
+ pruningPredicateText.contains("fun: StartsWith"),
+ pruningPredicateText)
Review Comment:
The test asserts STARTS_WITH pruning by searching for substrings in `pruningPredicates.mkString`, which depends on protobuf `toString` formatting and is likely to be brittle across protobuf/scalapb version or formatting changes. Prefer asserting on the structured `PhysicalExprNode` scalarFunction fields (name/fun) instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@auron.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AURON #2425] Support Iceberg STARTS_WITH pruning in native scan [auron]
Posted by "slfan1989 (via GitHub)" <gi...@apache.org>.
slfan1989 commented on PR #2426:
URL: https://github.com/apache/auron/pull/2426#issuecomment-5082138824
@weimingdiit Thanks for the contrubution! Merged into master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@auron.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AURON #2425] Support Iceberg STARTS_WITH pruning in native scan [auron]
Posted by "weimingdiit (via GitHub)" <gi...@apache.org>.
weimingdiit commented on PR #2426:
URL: https://github.com/apache/auron/pull/2426#issuecomment-5086730643
> @weimingdiit Thanks for the contrubution! Merged into master.
@slfan1989 Thanks for your review and merge.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@auron.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org