You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@polaris.apache.org by "george-zubrienko (via GitHub)" <gi...@apache.org> on 2025/03/18 13:27:47 UTC
[I] Cleanup tasks failing with value too long for type character varying(255) [polaris]
george-zubrienko opened a new issue, #1195:
URL: https://github.com/apache/polaris/issues/1195
### Describe the bug
When a table is dropped using DROP_WITH_PURGE (default in Trino connector), Polaris fails to drop the table data from the underlying storage, throwing this:
```
2025-03-18 12:46:24,760 INFO [org.apa.pol.ser.tas.TaskExecutorImpl] [,] [00000000000000000000000000000000,,0000000000000000,false] (executor-thread-1) Handling task entity id 155
2025-03-18 12:46:24,855 INFO [org.apa.pol.ser.tas.TableCleanupTaskHandler] [,] [00000000000000000000000000000000,,0000000000000000,false] (executor-thread-1) tableIdentifier=d365_eu.test metadataLocation=s3://.../metadata/00001-a8283615-e997-490a-8c48-28d75a02f29b.metadata.json Handling table metadata cleanup task
2025-03-18 12:46:24,856 INFO [org.apa.ice.CatalogUtil] [,] [00000000000000000000000000000000,,0000000000000000,false] (executor-thread-1) Loading custom FileIO implementation: org.apache.iceberg.aws.s3.S3FileIO
[EL Warning]: 2025-03-18 12:46:24.944--UnitOfWork(169674653)--Exception [EclipseLink-4002] (Eclipse Persistence Services - 4.0.5.v202412231137-a96b873527f305f932543045c8679bb1de8d3a43): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: org.postgresql.util.PSQLException: ERROR: value too long for type character varying(255)
Error Code: 0
Call: INSERT INTO ENTITIES (CATALOGID, ID, CREATETIMESTAMP, DROPTIMESTAMP, ENTITYVERSION, GRANTRECORDSVERSION, INTERNALPROPERTIES, LASTUPDATETIMESTAMP, NAME, PARENTID, PROPERTIES, PURGETIMESTAMP, SUBTYPECODE, TOPURGETIMESTAMP, TYPECODE, VERSION) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
bind => [16 parameters bound]
Query: InsertObjectQuery(org.apache.polaris.jpa.models.ModelEntity@ee2670d)
2025-03-18 12:46:26,945 WARN [org.apa.pol.ser.tas.TaskExecutorImpl] [,] [,,,] (executor-thread-1) Failed to handle task entity id 155: java.util.concurrent.CompletionException: jakarta.persistence.PersistenceException: Exception [EclipseLink-4002] (Eclipse Persistence Services - 4.0.5.v202412231137-a96b873527f305f932543045c8679bb1de8d3a43): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: org.postgresql.util.PSQLException: ERROR: value too long for type character varying(255)
Error Code: 0
Call: INSERT INTO ENTITIES (CATALOGID, ID, CREATETIMESTAMP, DROPTIMESTAMP, ENTITYVERSION, GRANTRECORDSVERSION, INTERNALPROPERTIES, LASTUPDATETIMESTAMP, NAME, PARENTID, PROPERTIES, PURGETIMESTAMP, SUBTYPECODE, TOPURGETIMESTAMP, TYPECODE, VERSION) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
bind => [16 parameters bound]
Query: InsertObjectQuery(org.apache.polaris.jpa.models.ModelEntity@ee2670d)
at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315)
at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:320)
at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1807)
at io.smallrye.context.impl.wrappers.SlowContextualRunnable.run(SlowContextualRunnable.java:19)
at org.jboss.threads.EnhancedViewExecutor$EnhancedViewExecutorRunnable.run(EnhancedViewExecutor.java:501)
at io.quarkus.vertx.core.runtime.VertxCoreRecorder$15.runWith(VertxCoreRecorder.java:643)
at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2675)
at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2654)
at org.jboss.threads.EnhancedQueueExecutor.runThreadBody(EnhancedQueueExecutor.java:1627)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1594)
at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:11)
at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:11)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: jakarta.persistence.PersistenceException: Exception [EclipseLink-4002] (Eclipse Persistence Services - 4.0.5.v202412231137-a96b873527f305f932543045c8679bb1de8d3a43): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: org.postgresql.util.PSQLException: ERROR: value too long for type character varying(255)
Error Code: 0
```
### To Reproduce
1. Have a working setup with Trino using Polaris as Iceberg REST catalog, S3 as a backing store, and make sure principal used has both TABLE_DROP and TABLE_WRITE_DATA permissions
2. Create a table from Trino: `create table mycatalog.myschema.test as select 'a' as colA`
3. Confirm table creation by running `select * from mycatalog.myschema.test`
4. Drop the table: `drop table mycatalog.myschema.test`
### Actual Behavior
Table is removed from catalog, but data is not purged from S3, error printed to logs
### Expected Behavior
Data is purged
### Additional context
_No response_
### System information
Polaris v0.9 with commit from Mar 17, built with Postgres Eclipse Link for container. Postgres database is Aurora Serverless 15.10, bootstrapped with Admin tool of the same version.
`./gradlew clean :polaris-quarkus-server:assemble -Dquarkus.container-image.build=true --no-build-cache -PeclipseLink=true -PeclipseLinkDeps=org.postgresql:postgresql:42.7.4`
`./gradlew clean :polaris-quarkus-admin:build -Dquarkus.container-image.build=true --no-build-cache -PeclipseLinkDeps=org.postgresql:postgresql:42.7.4`
Installed in EKS 1.29 via Helm chart built from the same commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Cleanup tasks failing with value too long for type character varying(255) [polaris]
Posted by "MonkeyCanCode (via GitHub)" <gi...@apache.org>.
MonkeyCanCode commented on issue #1195:
URL: https://github.com/apache/polaris/issues/1195#issuecomment-2735238983
> i see this concern here though : https://github.com/apache/polaris/blob/main/service/common/src/main/java/org/apache/polaris/service/task/TableCleanupTaskHandler.java#L180
>
> may be we should md5 the task name before inserting it, would prefer this rather than changing the name to TEXT.
>
> Lets see what other folks think cc [@eric-maynard](https://github.com/eric-maynard) [@collado-mike](https://github.com/collado-mike)
If we want to hash the task name, may be consider sha256 instead as md5 is disabled on system with FIPS enabled.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Cleanup tasks failing with value too long for type character varying(255) [polaris]
Posted by "george-zubrienko (via GitHub)" <gi...@apache.org>.
george-zubrienko commented on issue #1195:
URL: https://github.com/apache/polaris/issues/1195#issuecomment-2735543025
> internal properties and properties are already text, which version are you using https://github.com/apache/polaris/blob/main/extension/persistence/jpa-model/src/main/java/org/apache/polaris/jpa/models/ModelEntity.java#L80
>
> is the name problematic, its currently marked as string
>
The version I am using is noted in the issue description. Commit https://github.com/apache/polaris/commit/d33454874f69f952da2dfb301d0330d6e8d2296e
I bootstrapped the DB using admin tool built from the same commit. Might be one of the fields is indeed text - I haven't checked and applied mitigation to all, as this issue was present like 2 months ago.
Doesn't change the fact that table drop defacto doesn't work :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Cleanup tasks failing with value too long for type character varying(255) [polaris]
Posted by "dimas-b (via GitHub)" <gi...@apache.org>.
dimas-b commented on issue #1195:
URL: https://github.com/apache/polaris/issues/1195#issuecomment-4931257958
@lawrenz1 : There are other issues with purge, e.g. #2365
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Cleanup tasks failing with value too long for type character varying(255) [polaris]
Posted by "singhpk234 (via GitHub)" <gi...@apache.org>.
singhpk234 commented on issue #1195:
URL: https://github.com/apache/polaris/issues/1195#issuecomment-2734039899
internal properties and properties are already text, which version are you using https://github.com/apache/polaris/blob/main/extension/persistence/jpa-model/src/main/java/org/apache/polaris/jpa/models/ModelEntity.java#L80
is the name problematic, its currently marked as string
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Cleanup tasks failing with value too long for type character varying(255) [polaris]
Posted by "george-zubrienko (via GitHub)" <gi...@apache.org>.
george-zubrienko commented on issue #1195:
URL: https://github.com/apache/polaris/issues/1195#issuecomment-2735546184
> may be we should md5 the task name before inserting it, would prefer this rather than changing the name to TEXT.
In this case a migration script needs to be provided to hash existing data or type conversion will fail?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Cleanup tasks failing with value too long for type character varying(255) [polaris]
Posted by "singhpk234 (via GitHub)" <gi...@apache.org>.
singhpk234 commented on issue #1195:
URL: https://github.com/apache/polaris/issues/1195#issuecomment-2734056596
i see this concern here though : https://github.com/apache/polaris/blob/main/service/common/src/main/java/org/apache/polaris/service/task/TableCleanupTaskHandler.java#L180
may be we should md5 the task name before inserting it, would prefer this rather than changing the name to TEXT.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Cleanup tasks failing with value too long for type character varying(255) [polaris]
Posted by "singhpk234 (via GitHub)" <gi...@apache.org>.
singhpk234 commented on issue #1195:
URL: https://github.com/apache/polaris/issues/1195#issuecomment-2767652048
Trying to Handle this case for the JDBC impl I am working working on by making it TEXT.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org