You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@calcite.apache.org by "microbluey (via GitHub)" <gi...@apache.org> on 2026/07/27 08:38:37 UTC
[PR] [CALCITE-7487] ProjectJoinTransposeRule throws ArrayIndexOutOfBoundsException in PushProjector when a Join input has a zero-column row type [calcite]
microbluey opened a new pull request, #5125:
URL: https://github.com/apache/calcite/pull/5125
`PushProjector.locateAllRefs` contains a workaround, originally added for Fennel, that arbitrarily projects the first column of a `Join`/`SetOp` input when nothing else is projected from it:
```java
if (nProject == 0 && childPreserveExprs.isEmpty()) {
projRefs.set(0);
nProject = 1;
}
```
It assumes the input *has* a first column. That fails for a zero-column input — canonically DEE, the `Values` with an empty row type and a single empty tuple that is the identity for cross join (it arises when an `Aggregate` with `GROUP BY ()` has its output pruned to zero columns). The workaround then sets a bit past the input's fields, and `createProjectRefsAndExprs` uses it to index into an empty field list:
```
java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0
at org.apache.calcite.rel.rules.PushProjector.createProjectRefsAndExprs(PushProjector.java:523)
at org.apache.calcite.rel.rules.ProjectJoinTransposeRule.onMatch(ProjectJoinTransposeRule.java:117)
```
### Fix
Guard each workaround on the input it protects having at least one field: `nFields > 0` for the left, `nFieldsRight > 0` for the right. When there is genuinely no first column, the workaround is skipped instead of producing an out-of-range reference.
### Two deviations from the JIRA description, both verified
**1. The left-side case is not merely latent — it crashes too.** The ticket notes the symmetric `nProject == 0` path "has the same latent issue" but that only the right-side variant had been seen in practice. Putting DEE on the *left* of the cross join throws the identical exception, so there is a regression test for each direction.
**2. The suggested left-side guard `(nSysFields + nFields) > 0` would not be correct.** Given the field layout documented on `nFields`:
```
| nSysFields | nFields | nFieldsRight |
```
the left input has `nFields` fields, so folding `nSysFields` into the condition lets the workaround through when `nSysFields > 0 && nFields == 0`. `createProjectRefsAndExprs` uses `offset = nSysFields` for the left side, so `refIdx - offset` would then be negative — a different crash rather than a fix. Hence `nFields > 0`.
### Note on the resulting plan
Skipping the workaround means the pushed-down projection over the zero-column input is itself zero-column:
```
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(col1=[CAST($0):VARCHAR NOT NULL])
LogicalValues(tuples=[[{ 0 }]])
LogicalProject
LogicalValues(tuples=[[{ }]])
```
That is precisely what the Fennel workaround existed to avoid. Fennel has long been removed, and zero-column relational expressions are legal in Calcite today — DEE itself is one — so this seems right. Flagging it explicitly in case anyone disagrees, since it is the behavioural part of this change.
### Out of scope, but worth recording
`projRefs.set(0)` and `projRefs.set(nFields)` look like they should be `set(nSysFields)` and `set(nSysFields + nFields)`, since the left input's fields start after the system fields. Demonstrating that needs a Join that actually has system fields, and fixing it would change which column the workaround picks, so I have deliberately kept it out of this crash fix. Happy to file it separately if it is a real issue.
The reporter also wondered whether `ProjectCorrelateTransposeRule` has the same latent issue. It does construct a `PushProjector` (`ProjectCorrelateTransposeRule.java:79-80`), but the workaround is gated on `childRel instanceof Join || childRel instanceof SetOp`, and `Correlate extends BiRel` — it is neither. So that rule never reaches this code and is unaffected.
### Tests
Both new tests fail on current `main` with the `ArrayIndexOutOfBoundsException` above and pass with the fix. `RelOptRulesTest` is green (923 tests). Golden plans were produced via the standard `_actual.xml` mechanism.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7487] ProjectJoinTransposeRule throws ArrayIndexOutOfBoundsException in PushProjector when a Join input has a zero-column row type [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on PR #5125:
URL: https://github.com/apache/calcite/pull/5125#issuecomment-5248660787
@microbluey in order to merge this you need to squash your two commits into a single one
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7487] ProjectJoinTransposeRule throws ArrayIndexOutOfBoundsException in PushProjector when a Join input has a zero-column row type [calcite]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #5125:
URL: https://github.com/apache/calcite/pull/5125#issuecomment-5249704725
## [](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=5125) **Quality Gate passed**
Issues
 [1 New issue](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=5125&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=5125&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_calcite&pullRequest=5125&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [92.9% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=5125&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=5125&metric=new_duplicated_lines_density&view=list)
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=5125)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7487] ProjectJoinTransposeRule throws ArrayIndexOutOfBoundsException in PushProjector when a Join input has a zero-column row type [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on PR #5125:
URL: https://github.com/apache/calcite/pull/5125#issuecomment-5255494941
@microbluey the current main certainly builds fine. Can you elaborate about the failure on main that you are seeing? Maybe you can file an issue about it if it is a bug?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7487] ProjectJoinTransposeRule throws ArrayIndexOutOfBoundsException in PushProjector when a Join input has a zero-column row type [calcite]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #5125:
URL: https://github.com/apache/calcite/pull/5125#issuecomment-5089348054
## [](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=5125) **Quality Gate passed**
Issues
 [1 New issue](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=5125&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=5125&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_calcite&pullRequest=5125&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [92.9% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=5125&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=5125&metric=new_duplicated_lines_density&view=list)
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=5125)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7487] ProjectJoinTransposeRule throws ArrayIndexOutOfBoundsException in PushProjector when a Join input has a zero-column row type [calcite]
Posted by "microbluey (via GitHub)" <gi...@apache.org>.
microbluey commented on PR #5125:
URL: https://github.com/apache/calcite/pull/5125#issuecomment-5249536115
Squashed into one commit, and rebased onto current main while I was there (main had moved 47 commits, some touching these files). Both regression tests still pass and `:core:style` is clean.
One thing worth flagging: `RelOptRulesTest` already has a failure on current main, independent of this PR — a plan diff under `testDecorrelateProjectWithMultiKeyAndFetchOne1` plus a missing `testEmptyCalc` golden. I reproduced it on a clean `origin/main` checkout with none of my changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7487] ProjectJoinTransposeRule throws ArrayIndexOutOfBoundsException in PushProjector when a Join input has a zero-column row type [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on code in PR #5125:
URL: https://github.com/apache/calcite/pull/5125#discussion_r3659043795
##########
core/src/test/java/org/apache/calcite/test/RelOptRulesTest.java:
##########
@@ -1559,6 +1559,56 @@ private void checkSemiOrAntiJoinProjectTranspose(JoinRelType type) {
checkJoinProjectTransposeDoesNotMatch(JoinRelType.LEFT_MARK);
}
+ /** Test case for
+ * <a href="https://issues.apache.org/jira/browse/CALCITE-7487">[CALCITE-7487]
+ * ProjectJoinTransposeRule throws ArrayIndexOutOfBoundsException in
+ * PushProjector when a Join input has a zero-column row type</a>. */
+ @Test void testProjectJoinTransposeWithZeroColumnRightInput() {
+ relFn(b -> zeroColumnJoinInputRelFn(b, false))
+ .withRule(CoreRules.PROJECT_JOIN_TRANSPOSE).check();
+ }
+
+ /** Test case for
+ * <a href="https://issues.apache.org/jira/browse/CALCITE-7487">[CALCITE-7487]
+ * ProjectJoinTransposeRule throws ArrayIndexOutOfBoundsException in
+ * PushProjector when a Join input has a zero-column row type</a>. */
+ @Test void testProjectJoinTransposeWithZeroColumnLeftInput() {
+ relFn(b -> zeroColumnJoinInputRelFn(b, true))
+ .withRule(CoreRules.PROJECT_JOIN_TRANSPOSE).check();
+ }
+
+ /** Builds {@code Project(CAST(col1))} over a cross join in which one input is
+ * DEE -- a {@link org.apache.calcite.rel.core.Values} with an empty row type,
+ * the identity for cross join. The project must be non-identity, otherwise
+ * {@link RelBuilder} collapses it away and the rule never fires. */
+ private static RelNode zeroColumnJoinInputRelFn(RelBuilder b,
Review Comment:
Can this be reproduced with a quidem test?
Not all plans that the builder can build can surface from a real program.
##########
core/src/main/java/org/apache/calcite/rel/rules/PushProjector.java:
##########
@@ -457,13 +457,21 @@ public boolean locateAllRefs() {
|| (childRel instanceof SetOp)) {
// if nothing is projected from the children, arbitrarily project
// the first columns; this is necessary since Fennel doesn't
- // handle 0-column projections
- if (nProject == 0 && childPreserveExprs.isEmpty()) {
+ // handle 0-column projections.
+ //
+ // An input may legitimately have a zero-column row type -- for
+ // example DEE, the empty-row-type Values that is the identity for
Review Comment:
What is DEE?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7487] ProjectJoinTransposeRule throws ArrayIndexOutOfBoundsException in PushProjector when a Join input has a zero-column row type [calcite]
Posted by "microbluey (via GitHub)" <gi...@apache.org>.
microbluey commented on PR #5125:
URL: https://github.com/apache/calcite/pull/5125#issuecomment-5094221588
Thanks for looking. Both answered, and I've pushed a commit for the first.
**"What is DEE?"** — fair challenge: the abbreviation appears nowhere else in the codebase, so it shouldn't be in a comment. It's the standard relational-algebra name for the relation with no attributes and one tuple (the identity for cross join; DUM is its empty counterpart), but I've replaced it with a plain description — "a Values with an empty row type and a single empty tuple, which returns one row with zero columns" — in `269e562`.
**"Can this be reproduced with a quidem test?"** — I tried, and I don't think it can, which I should have said in the description rather than leaving you to ask.
You're right that not every builder-constructible plan surfaces from a real program. I attempted the SQL routes to a zero-column join input:
```
-- (select) as u -> parse error, not valid syntax
select cast(a.x as varchar) from (values (1)) as a(x)
cross join (select count(*) from (values (1),(2))) as b; -- runs fine
select cast(a.x as varchar) as c from (values (1)) as a(x)
cross join (select 1 from (values (1),(2)) group by ()) as b; -- runs fine
```
Both execute correctly. The aggregate keeps a column in the plan the validator produces, so the workaround always has a first column to fall back on and the crash never triggers. `RelFieldTrimmer` deliberately avoids handing anything a zero-column row type for the same historical reason as this workaround (see the "Fennel abhors an empty row type" comment at `RelFieldTrimmer.java:1031`), which is presumably why no SQL path reaches it.
So the reachable route is via `RelBuilder` or an equivalent frontend rather than Calcite's own SQL: the reporter hit it through substrait-java, where an `Aggregate(GROUP BY (), measures)` with `Rel.Emit{outputMapping=[]}` yields exactly this shape on one side of a cross join. Since `PushProjector` guards defensively elsewhere and the fix is two bounds checks, guarding here seemed worth it even though Calcite's own parser can't get there — but if you'd rather this be fixed at the frontend, or declined as unreachable, that's a reasonable call and I'm happy to close it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7487] ProjectJoinTransposeRule throws ArrayIndexOutOfBoundsException in PushProjector when a Join input has a zero-column row type [calcite]
Posted by "microbluey (via GitHub)" <gi...@apache.org>.
microbluey commented on PR #5125:
URL: https://github.com/apache/calcite/pull/5125#issuecomment-5260004210
You're right, and I'm sorry for the noise — there is no bug on main. The failure was mine.
`core/build/diffrepo/.../RelOptRulesTest_actual.xml` was a leftover from 27 July, and `DiffRepository` compares the reference XML against that file. Main had gained 9 new `TestCase` entries since then (including `testEmptyCalc`), so the stale actual was missing them and the comparison failed. I had also run a `--tests` subset earlier, which is what left a partial file behind.
After `rm -rf core/build/diffrepo core/build/test-results/test` and a `--rerun-tasks` run:
- `origin/main` — `BUILD SUCCESSFUL`, no failures
- this branch — 930 tests, 0 failures, 0 errors
So nothing to file. Sorry for holding up the merge over it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7487] ProjectJoinTransposeRule throws ArrayIndexOutOfBoundsException in PushProjector when a Join input has a zero-column row type [calcite]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #5125:
URL: https://github.com/apache/calcite/pull/5125#issuecomment-5094469382
## [](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=5125) **Quality Gate passed**
Issues
 [1 New issue](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=5125&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=5125&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_calcite&pullRequest=5125&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [92.9% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=5125&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=5125&metric=new_duplicated_lines_density&view=list)
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=5125)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7487] ProjectJoinTransposeRule throws ArrayIndexOutOfBoundsException in PushProjector when a Join input has a zero-column row type [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on PR #5125:
URL: https://github.com/apache/calcite/pull/5125#issuecomment-5107654583
I think you can squash the commits for merging
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7487] ProjectJoinTransposeRule throws ArrayIndexOutOfBoundsException in PushProjector when a Join input has a zero-column row type [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on PR #5125:
URL: https://github.com/apache/calcite/pull/5125#issuecomment-5259180835
I am waiting the answer about the additional bug before merging, I don't want to lose it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7487] ProjectJoinTransposeRule throws ArrayIndexOutOfBoundsException in PushProjector when a Join input has a zero-column row type [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu merged PR #5125:
URL: https://github.com/apache/calcite/pull/5125
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org