You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@auron.apache.org by "weiqingy (via GitHub)" <gi...@apache.org> on 2026/06/02 07:25:16 UTC
[PR] [AURON #1861] Convert comparison operators and LIKE to native operators [auron]
weiqingy opened a new pull request, #2298:
URL: https://github.com/apache/auron/pull/2298
# Which issue does this PR close?
Closes #1861
# Rationale for this change
Flink Calc expressions that use comparison operators (`=`, `<>`, `>`, `<`, `>=`, `<=`) or `LIKE`
could not run on the Auron native engine — the Flink `RexCallConverter` recognized only arithmetic and
`CAST`, so any Calc containing a comparison fell back to the Flink engine. Comparisons are the backbone
of `WHERE`/`JOIN`/`CASE` predicates, so without them most real queries cannot be accelerated.
# What changes are included in this PR?
Extends the existing `RexCallConverter` (no proto, Rust, or dependency changes — the native plumbing
already exists for all of these operators):
- **Comparison operators** → `PhysicalBinaryExprNode` with op strings `Eq` / `NotEq` / `Gt` / `Lt` /
`GtEq` / `LtEq`. Operands are promoted to a common type before conversion (the native comparison
kernel requires matching operand types and performs no implicit coercion); the boolean result needs
no output cast.
- **`LIKE`** → the dedicated `PhysicalLikeExprNode`. `LIKE ... ESCAPE c` (explicit, non-default escape)
is reported unsupported so the Calc falls back, since the native like node has no escape field.
`NOT LIKE` is represented by Calcite as `NOT(LIKE(...))`; it converts end-to-end once the logical
`NOT` converter is in place, and falls back to Flink until then.
Two commits: one for the six comparison operators, one for `LIKE`.
A few existing tests used `EQUALS` as a known-unsupported operator to exercise the fallback path; now
that comparisons are supported, they use `SIMILAR TO` (still unsupported) to keep testing fallback.
# Are there any user-facing changes?
Flink queries whose Calc predicates use these comparison operators or `LIKE` now execute on the native
engine instead of falling back to Flink. No API or configuration change.
# How was this patch tested?
New unit tests in `RexCallConverterTest` cover each comparison operator's native op string, operand
type promotion, `LIKE`, the `NOT(LIKE)` fallback, and the explicit-`ESCAPE` fallback. The full
`auron-flink-planner` module test suite passes (82 tests), with 0 Checkstyle violations.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@auron.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AURON #1861] Convert comparison operators and LIKE to native operators [auron]
Posted by "weiqingy (via GitHub)" <gi...@apache.org>.
weiqingy commented on PR #2298:
URL: https://github.com/apache/auron/pull/2298#issuecomment-4607262149
The 1 failing check is unrelated to this PR:
- **Test spark-4.0 JDK21 Scala-2.13 / Build Auron JAR**: `JVM/RPC crash in spark-4.0 integration tests` — the `auron-spark-tests-spark40` module died mid-suite when the driver's in-test Spark REPL class server (`spark://...:43973`) became unreachable (`RemoteClassLoaderError` / `event executor terminated` / connect-refused), followed by a fatal `SIGSEGV` in `libjvm.so` (core dump + adoptium bug-report footer). This PR changes only Flink planner files under `auron-flink-extension/` — no Spark, native-engine, or proto code — and every other Spark build (3.0–3.5, 4.1) plus Test Flink 1.18 passed on the same commit. Evidence: [failing job](https://github.com/apache/auron/actions/runs/26842352482/job/79152589175).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@auron.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AURON #1861] Convert comparison operators and LIKE to native operators [auron]
Posted by "Tartarus0zm (via GitHub)" <gi...@apache.org>.
Tartarus0zm merged PR #2298:
URL: https://github.com/apache/auron/pull/2298
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@auron.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AURON #1861] Convert comparison operators and LIKE to native operators [auron]
Posted by "weiqingy (via GitHub)" <gi...@apache.org>.
weiqingy commented on PR #2298:
URL: https://github.com/apache/auron/pull/2298#issuecomment-4609952449
> Hi @weiqingy #1860 is merged, please resolve the merge conflicts.
Hi @Tartarus0zm, I’ve updated the PR to resolve the conflicts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@auron.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AURON #1861] Convert comparison operators and LIKE to native operators [auron]
Posted by "weiqingy (via GitHub)" <gi...@apache.org>.
weiqingy commented on PR #2298:
URL: https://github.com/apache/auron/pull/2298#issuecomment-4606345335
> @weiqingy Thanks for your contribution! The Flink Calc operator integration flow is already working end-to-end. We can now add full query tests in ITCase.
Thanks for the review! Added 9 end-to-end SQL ITCase tests in `AuronFlinkCalcITCase` (commit `8c886da7`).
They submit real queries over `T1` through `StreamTableEnvironment` and assert the result rows, covering all six comparison operators (`=`, `<>`, `>`, `<`, `>=`, `<=`) in `WHERE` filters, a comparison used directly in a Boolean projection, an INT-vs-DOUBLE filter that exercises operand promotion, and a `LIKE` filter.
The run logs confirm these execute natively — e.g. `FilterExec [string@0 LIKE Comment%]` and `ProjectExec [int@0 > 1]` over the FFIReader — so the Calc integration flow is genuinely working end-to-end for these operators.
I also refreshed the now-outdated comment on `AuronCalcRewriteITCase.testFilterAndProjectEndToEnd`, which still described comparisons as unsupported / falling back. Full planner suite is green (91 tests) with 0 Checkstyle violations.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@auron.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AURON #1861] Convert comparison operators and LIKE to native operators [auron]
Posted by "Tartarus0zm (via GitHub)" <gi...@apache.org>.
Tartarus0zm commented on PR #2298:
URL: https://github.com/apache/auron/pull/2298#issuecomment-4609593908
Hi @weiqingy #1860 is merged, please resolve the merge conflicts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@auron.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org