You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "wombatu-kun (via GitHub)" <gi...@apache.org> on 2026/05/19 07:37:14 UTC
[PR] Spark: Re-enable subquery-expression view tests for the Hive view catalog [iceberg]
wombatu-kun opened a new pull request, #16416:
URL: https://github.com/apache/iceberg/pull/16416
Closes #15053
## What
`TestViews#createViewWithSubqueryExpressionInFilterThatIsRewritten` and `...InQueryThatIsRewritten` were skipped for the Hive view catalog (`SPARK_WITH_HIVE_VIEWS`) by an `assumeThat` that blamed a `FileInputFormat` instantiation error. This re-enables both tests for the Hive view catalog on Spark 3.5, 4.0 and 4.1.
## Why
Investigation (see #15053) shows the Hive-backed subquery view itself works — creating the view, reading it, and the catalog/namespace SQL rewrite all pass. The only failing step was the cross-catalog negative assertion: it switches to `spark_catalog` and expects the unqualified table to be unresolvable. That premise does not hold for the Hive view catalog, because it shares its Hive Metastore with Spark's built-in `spark_catalog` — the Iceberg table is resolvable there, and a native Hive read of it then fails at execution time (Iceberg deliberately writes a placeholder InputFormat when `engine.hive.enabled` is false, since a native read of an Iceberg table would otherwise return silently-wrong results). So this is a test-correctness issue, not an Iceberg view defect, and the abstract-InputFormat behavior is intentional and out of scope.
## How
- Remove the blanket `assumeThat(...).isNotEqualTo("hive")` so both tests run for the Hive view catalog.
- Branch the cross-catalog negative assertion: for the Hive view catalog assert the raw SQL fails at execution with `SparkException` (table found in the shared metastore but not natively readable); otherwise keep the existing `AnalysisException` not-found assertion.
- Applied identically to `spark/v3.5`, `spark/v4.0` and `spark/v4.1`. (`spark/v3.4` never had the Hive view catalog parameter, so it is unchanged.)
The Hive-specific assertion uses the top-level `org.apache.spark.SparkException`, which is stable across Spark 3.5/4.0/4.1 (the underlying cause differs by version and is intentionally not asserted on).
## Testing
`createViewWithSubqueryExpressionInFilterThatIsRewritten` and `createViewWithSubqueryExpressionInQueryThatIsRewritten` now pass for all parameters, including `SPARK_WITH_HIVE_VIEWS`, on Spark 3.5, 4.0 and 4.1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark: Re-enable subquery-expression view tests for the Hive view catalog [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #16416:
URL: https://github.com/apache/iceberg/pull/16416#issuecomment-6103782561
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark: Re-enable subquery-expression view tests for the Hive view catalog [iceberg]
Posted by "wombatu-kun (via GitHub) via issues" <is...@iceberg.apache.org>.
wombatu-kun closed pull request #16416: Spark: Re-enable subquery-expression view tests for the Hive view catalog
URL: https://github.com/apache/iceberg/pull/16416
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark: Re-enable subquery-expression view tests for the Hive view catalog [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #16416:
URL: https://github.com/apache/iceberg/pull/16416#issuecomment-4755743404
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark: Re-enable subquery-expression view tests for the Hive view catalog [iceberg]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on PR #16416:
URL: https://github.com/apache/iceberg/pull/16416#issuecomment-4755802725
not stale
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org