You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "manuzhang (via GitHub)" <gi...@apache.org> on 2026/09/14 14:03:33 UTC
[I] Flaky test: TestViews.readFromViewReferencingTempFunction fails with ROUTINE_ALREADY_EXISTS [iceberg]
manuzhang opened a new issue, #18103:
URL: https://github.com/apache/iceberg/issues/18103
### Apache Iceberg version
main (development)
### Query engine
Spark
### Please describe the bug 🐞
`TestViews.readFromViewReferencingTempFunction` failed on the Spark 4.2 extensions tests in a `main` push build:
https://github.com/apache/iceberg/actions/runs/34809521107/job/103867881304
```
TestViews > readFromViewReferencingTempFunction() > catalogName = spark_hive_with_views, implementation = org.apache.iceberg.spark.SparkCatalog, config = {type=hive, default-namespace=default, cache-enabled=false} FAILED
org.apache.spark.sql.catalyst.analysis.FunctionAlreadyExistsException: [ROUTINE_ALREADY_EXISTS] Cannot create the routine `test_avg919406` because a routine of that name already exists.
Choose a different name, drop or replace the existing routine, or add the IF NOT EXISTS clause to tolerate a pre-existing routine. SQLSTATE: 42723
at app//org.apache.spark.sql.errors.QueryCompilationErrors$.functionAlreadyExistsError(QueryCompilationErrors.scala:1309)
at app//org.apache.spark.sql.execution.command.CreateFunctionCommand.run(functions.scala:68)
...
at app//org.apache.iceberg.spark.TestBase.sql(TestBase.java:131)
at app//org.apache.iceberg.spark.extensions.TestViews.readFromViewReferencingTempFunction(TestViews.java:614)
```
#### Likely cause
The temp function name is generated with `viewName("test_avg")`, which appends `new Random().nextInt(1000000)`. The temp function is never dropped.
`TestViews` runs each test once per catalog parameter (`spark_with_views`, `spark_catalog`, `spark_hive_with_views`) against the same `SparkSession`. The temp functions created by the earlier parameter runs therefore stay registered. When the random suffix repeats, `CREATE TEMPORARY FUNCTION` fails.
Spark 4.2's `CreateFunctionCommand` throws this error only when `SessionCatalog.isRegisteredFunction` finds the unqualified name in the session's temp function registry. It does not check the metastore, so the name must have been registered earlier in the same session. The collision is rare, which makes the test flaky.
The same pattern (random suffix, no `DROP TEMPORARY FUNCTION`) exists in `readFromViewReferencingTempFunction`, `createViewReferencingTempFunction`, and `createViewReferencingQualifiedTempFunction`. It is present in the Spark 3.5, 4.0, 4.1 and 4.2 versions of `TestViews`.
#### Possible fix
- Drop the temp function in a `finally` block (`DROP TEMPORARY FUNCTION IF EXISTS ...`), as `createViewReferencingTemporaryVariable` already does for its temporary variable; and/or
- generate collision-free names in `viewName()` (e.g. an incrementing counter or a UUID-based suffix) instead of `new Random().nextInt(1000000)`.
### Willingness to contribute
- [ ] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- [x] I cannot contribute a fix for this bug at this time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Flaky test: TestViews.readFromViewReferencingTempFunction fails with ROUTINE_ALREADY_EXISTS [iceberg]
Posted by "zhang-arvin (via GitHub)" <gi...@apache.org>.
zhang-arvin commented on issue #18103:
URL: https://github.com/apache/iceberg/issues/18103#issuecomment-5666888869
I will work on this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Flaky test: TestViews.readFromViewReferencingTempFunction fails with ROUTINE_ALREADY_EXISTS [iceberg]
Posted by "HesandaLiyanage (via GitHub)" <gi...@apache.org>.
HesandaLiyanage commented on issue #18103:
URL: https://github.com/apache/iceberg/issues/18103#issuecomment-5666850415
@manuzhang Hi , I would like to work on this issue. Can you assign this to me ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org