You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/12/19 16:49:13 UTC
[GitHub] [iceberg] MurderWind opened a new issue, #6455: flink streaming query Cannot get a client from a closed pool
MurderWind opened a new issue, #6455:
URL: https://github.com/apache/iceberg/issues/6455
### Apache Iceberg version
1.1.0 (latest release)
### Query engine
Flink
### Please describe the bug 🐞
flink: 1.16.0
- Start scheduling with scheduling strategy， i get errors when Job Flink Streaming Job running . of course . i use catalog-impl is "org.apache.iceberg.jdbc.JdbcCatalog"
40601 [SourceCoordinator-Source: Summer_Iceberg] ERROR org.apache.iceberg.flink.source.enumerator.ContinuousIcebergEnumerator - Failed to discover new splits
java.lang.IllegalStateException: Cannot get a client from a closed pool
at org.apache.iceberg.relocated.com.google.common.base.Preconditions.checkState(Preconditions.java:502)
at org.apache.iceberg.ClientPoolImpl.get(ClientPoolImpl.java:118)
at org.apache.iceberg.ClientPoolImpl.run(ClientPoolImpl.java:56)
at org.apache.iceberg.ClientPoolImpl.run(ClientPoolImpl.java:51)
at org.apache.iceberg.jdbc.JdbcTableOperations.getTable(JdbcTableOperations.java:228)
at org.apache.iceberg.jdbc.JdbcTableOperations.doRefresh(JdbcTableOperations.java:74)
at org.apache.iceberg.BaseMetastoreTableOperations.refresh(BaseMetastoreTableOperations.java:97)
at org.apache.iceberg.BaseTable.refresh(BaseTable.java:68)
at org.apache.iceberg.flink.source.enumerator.ContinuousSplitPlannerImpl.planSplits(ContinuousSplitPlannerImpl.java:73)
at org.apache.iceberg.flink.source.enumerator.ContinuousIcebergEnumerator.discoverSplits(ContinuousIcebergEnumerator.java:95)
at org.apache.flink.runtime.source.coordinator.ExecutorNotifier.lambda$notifyReadyAsync$5(ExecutorNotifier.java:130)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset$$$capture(FutureTask.java:308)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] xuzhiwen1255 commented on issue #6455: flink streaming query Cannot get a client from a closed pool
Posted by GitBox <gi...@apache.org>.
xuzhiwen1255 commented on issue #6455:
URL: https://github.com/apache/iceberg/issues/6455#issuecomment-1386445704
I found the cause of the problem and tried to solve it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] github-actions[bot] commented on issue #6455: flink streaming query Cannot get a client from a closed pool
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #6455:
URL: https://github.com/apache/iceberg/issues/6455#issuecomment-1639073830
This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] github-actions[bot] closed issue #6455: flink streaming query Cannot get a client from a closed pool
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #6455: flink streaming query Cannot get a client from a closed pool
URL: https://github.com/apache/iceberg/issues/6455
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] MurderWind commented on issue #6455: flink streaming query Cannot get a client from a closed pool
Posted by GitBox <gi...@apache.org>.
MurderWind commented on issue #6455:
URL: https://github.com/apache/iceberg/issues/6455#issuecomment-1359688519
> @MurderWind if you can reproduce it in IDE, you may be able to troubleshoot it with debugger.
>
> `IcebergSource` should use a new thread pool
>
> ```
> public ContinuousSplitPlannerImpl(Table table, ScanContext scanContext, String threadName) {
> this.table = table;
> this.scanContext = scanContext;
> this.isSharedPool = threadName == null;
> this.workerPool =
> isSharedPool
> ? ThreadPools.getWorkerPool()
> : ThreadPools.newWorkerPool(
> "iceberg-plan-worker-pool-" + threadName, scanContext.planParallelism());
> }
> ```
>
> As the threadName is non-null passed in from `IcebergSource`
>
> ```
> // Ideally, operatorId should be used as the threadPoolName as Flink guarantees its uniqueness
> // within a job. SplitEnumeratorContext doesn't expose the OperatorCoordinator.Context, which
> // would contain the OperatorID. Need to discuss with Flink community whether it is ok to expose
> // a public API like the protected method "OperatorCoordinator.Context getCoordinatorContext()"
> // from SourceCoordinatorContext implementation. For now, <table name>-<random UUID> is used as
> // the unique thread pool name.
> return lazyTable().name() + "-" + UUID.randomUUID();
> }
> ```
actually. i i found the bug when org.apache.iceberg.jdbc.JdbcTableOperations#connections connections properties closed turn true ,it is happened. but i don't find why closed turns true.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] MurderWind commented on issue #6455: flink streaming query Cannot get a client from a closed pool
Posted by GitBox <gi...@apache.org>.
MurderWind commented on issue #6455:
URL: https://github.com/apache/iceberg/issues/6455#issuecomment-1359687357
> // Ideally, operatorId should be used as the threadPoolName as Flink guarantees its uniqueness
> // within a job. SplitEnumeratorContext doesn't expose the OperatorCoordinator.Context, which
> // would contain the OperatorID. Need to discuss with Flink community whether it is ok to expose
> // a public API like the protected method "OperatorCoordinator.Context getCoordinatorContext()"
> // from SourceCoordinatorContext implementation. For now, <table name>-<random UUID> is used as
> // the unique thread pool name.
actually. i i found the bug when org.apache.iceberg.jdbc.JdbcTableOperations#connections connections properties closed turn true ,it is happened. but i don't find why closed turns true.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] MurderWind commented on issue #6455: flink streaming query Cannot get a client from a closed pool
Posted by GitBox <gi...@apache.org>.
MurderWind commented on issue #6455:
URL: https://github.com/apache/iceberg/issues/6455#issuecomment-1360718909
> @MurderWind if you can reproduce it in IDE, you may be able to troubleshoot it with debugger.
>
> `IcebergSource` should use a new thread pool
>
> ```
> public ContinuousSplitPlannerImpl(Table table, ScanContext scanContext, String threadName) {
> this.table = table;
> this.scanContext = scanContext;
> this.isSharedPool = threadName == null;
> this.workerPool =
> isSharedPool
> ? ThreadPools.getWorkerPool()
> : ThreadPools.newWorkerPool(
> "iceberg-plan-worker-pool-" + threadName, scanContext.planParallelism());
> }
> ```
>
> As the threadName is non-null passed in from `IcebergSource`
>
> ```
> // Ideally, operatorId should be used as the threadPoolName as Flink guarantees its uniqueness
> // within a job. SplitEnumeratorContext doesn't expose the OperatorCoordinator.Context, which
> // would contain the OperatorID. Need to discuss with Flink community whether it is ok to expose
> // a public API like the protected method "OperatorCoordinator.Context getCoordinatorContext()"
> // from SourceCoordinatorContext implementation. For now, <table name>-<random UUID> is used as
> // the unique thread pool name.
> return lazyTable().name() + "-" + UUID.randomUUID();
> }
> ```
i found the issues .
` protected JdbcTableOperations(
JdbcClientPool dbConnPool,
FileIO fileIO,
String catalogName,
TableIdentifier tableIdentifier,
Map<String, String> catalogProperties) {
this.catalogName = catalogName;
this.tableIdentifier = tableIdentifier;
this.fileIO = fileIO;
this.connections = dbConnPool;
this.catalogProperties = catalogProperties;
}`
the dbConnPool has initialize. but org.apache.iceberg.jdbc.JdbcCatalog#close this.closed =true . when org.apache.iceberg.ClientPoolImpl#get Preconditions.checkState(!this.closed, "Cannot get a client from a closed pool");
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] stevenzwu commented on issue #6455: flink streaming query Cannot get a client from a closed pool
Posted by GitBox <gi...@apache.org>.
stevenzwu commented on issue #6455:
URL: https://github.com/apache/iceberg/issues/6455#issuecomment-1358855539
@MurderWind if you can reproduce it in IDE, you may be able to troubleshoot it with debugger.
`IcebergSource` should use a new thread pool
```
public ContinuousSplitPlannerImpl(Table table, ScanContext scanContext, String threadName) {
this.table = table;
this.scanContext = scanContext;
this.isSharedPool = threadName == null;
this.workerPool =
isSharedPool
? ThreadPools.getWorkerPool()
: ThreadPools.newWorkerPool(
"iceberg-plan-worker-pool-" + threadName, scanContext.planParallelism());
}
```
As the threadName is non-null passed in from `IcebergSource`
```
// Ideally, operatorId should be used as the threadPoolName as Flink guarantees its uniqueness
// within a job. SplitEnumeratorContext doesn't expose the OperatorCoordinator.Context, which
// would contain the OperatorID. Need to discuss with Flink community whether it is ok to expose
// a public API like the protected method "OperatorCoordinator.Context getCoordinatorContext()"
// from SourceCoordinatorContext implementation. For now, <table name>-<random UUID> is used as
// the unique thread pool name.
return lazyTable().name() + "-" + UUID.randomUUID();
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] github-actions[bot] commented on issue #6455: flink streaming query Cannot get a client from a closed pool
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #6455:
URL: https://github.com/apache/iceberg/issues/6455#issuecomment-1661270883
This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org