You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/11/28 09:25:59 UTC
[GitHub] [hudi] aBlackAnt opened a new issue, #7314: [SUPPORT] Caused by: java.lang.IllegalArgumentException: ALREADY_ACQUIRED
aBlackAnt opened a new issue, #7314:
URL: https://github.com/apache/hudi/issues/7314
**Describe the problem you faced**
A clear and concise description of the problem.
**To Reproduce**
1.create table
```
CREATE TABLE if not exists test_hudi_flink_mor_occ (
uuid VARCHAR(200) PRIMARY KEY NOT ENFORCED,
name VARCHAR(100),
age INT,
ts TIMESTAMP(0),
`partition` VARCHAR(20)
)
PARTITIONED BY (`partition`
)
WITH (
'connector' = 'hudi',
'path' = '/project/ccr_ai_upgrade/ccr_ai_upgrade/aiui/flink/demo/test_hudi_flink_mor_occ',
'table.type' = 'MERGE_ON_READ',
'hoodie.datasource.write.keygenerator.class' = 'org.apache.hudi.keygen.ComplexAvroKeyGenerator',
'hoodie.datasource.write.recordkey.field' = 'uuid',
'write.precombine.field' = 'ts',
'hoodie.datasource.write.hive_style_partitioning' = 'true',
'hoodie.write.concurrency.mode' = 'optimistic_concurrency_control',
'hoodie.cleaner.policy.failed.writes' = 'LAZY',
'hoodie.write.lock.provider' = 'org.apache.hudi.hive.transaction.lock.HiveMetastoreBasedLockProvider',
'hoodie.write.lock.hivemetastore.database' = 'hive',
'hoodie.write.lock.hivemetastore.table' = 'test_hudi_flink_mor_occ_lock'
)
```
2.flink sql insert
```
insert into test_hudi_flink_mor_occ values ('a','liangji',18,TIMESTAMP '2022-11-20 18:00:00','2022-11-20')
```
**Environment Description**
* Hudi version :0.12.1
* Flink version :1.14.0
* Hadoop version :2.6.0
**Stacktrace**
```org.apache.flink.util.FlinkException: Global failure triggered by OperatorCoordinator for 'stream_write: test_hudi_flink_mor_occ' (operator 905882b372467a732faa32d0b002df8d).
at org.apache.flink.runtime.operators.coordination.OperatorCoordinatorHolder$LazyInitializedCoordinatorContext.failJob(OperatorCoordinatorHolder.java:545)
at org.apache.hudi.sink.StreamWriteOperatorCoordinator.lambda$start$0(StreamWriteOperatorCoordinator.java:187)
at org.apache.hudi.sink.utils.NonThrownExecutor.handleException(NonThrownExecutor.java:146)
at org.apache.hudi.sink.utils.NonThrownExecutor.lambda$wrapAction$0(NonThrownExecutor.java:133)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
at java.util.concurrent.FutureTask.run(FutureTask.java)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.hudi.exception.HoodieException: Executor executes action [handle end input event for instant 20221128171431329] error
... 9 more
Caused by: java.lang.IllegalArgumentException: ALREADY_ACQUIRED
at org.apache.hudi.common.util.ValidationUtils.checkArgument(ValidationUtils.java:40)
at org.apache.hudi.hive.transaction.lock.HiveMetastoreBasedLockProvider.acquireLock(HiveMetastoreBasedLockProvider.java:137)
at org.apache.hudi.hive.transaction.lock.HiveMetastoreBasedLockProvider.tryLock(HiveMetastoreBasedLockProvider.java:113)
at org.apache.hudi.client.transaction.lock.LockManager.lock(LockManager.java:73)
at org.apache.hudi.client.HoodieFlinkWriteClient.writeTableMetadata(HoodieFlinkWriteClient.java:280)
at org.apache.hudi.client.BaseHoodieWriteClient.commit(BaseHoodieWriteClient.java:270)
at org.apache.hudi.client.BaseHoodieWriteClient.commitStats(BaseHoodieWriteClient.java:235)
at org.apache.hudi.client.HoodieFlinkWriteClient.commit(HoodieFlinkWriteClient.java:125)
at org.apache.hudi.sink.StreamWriteOperatorCoordinator.doCommit(StreamWriteOperatorCoordinator.java:530)
at org.apache.hudi.sink.StreamWriteOperatorCoordinator.commitInstant(StreamWriteOperatorCoordinator.java:506)
at org.apache.hudi.sink.StreamWriteOperatorCoordinator.commitInstant(StreamWriteOperatorCoordinator.java:477)
at org.apache.hudi.sink.StreamWriteOperatorCoordinator.handleEndInputEvent(StreamWriteOperatorCoordinator.java:420)
at org.apache.hudi.sink.StreamWriteOperatorCoordinator.lambda$handleEventFromOperator$3(StreamWriteOperatorCoordinator.java:278)
at org.apache.hudi.sink.utils.NonThrownExecutor.lambda$wrapAction$0(NonThrownExecutor.java:130)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] just-JL commented on issue #7314: [SUPPORT] Caused by: java.lang.IllegalArgumentException: ALREADY_ACQUIRED
Posted by GitBox <gi...@apache.org>.
just-JL commented on issue #7314:
URL: https://github.com/apache/hudi/issues/7314#issuecomment-1329991424
My hudi version is 0.12.1, [https://github.com/apache/hudi/pull/6272](url) is already merged. So it can not resolve my problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] just-JL commented on issue #7314: [SUPPORT] Caused by: java.lang.IllegalArgumentException: ALREADY_ACQUIRED
Posted by GitBox <gi...@apache.org>.
just-JL commented on issue #7314:
URL: https://github.com/apache/hudi/issues/7314#issuecomment-1330031006
>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] just-JL commented on issue #7314: [SUPPORT] Caused by: java.lang.IllegalArgumentException: ALREADY_ACQUIRED
Posted by GitBox <gi...@apache.org>.
just-JL commented on issue #7314:
URL: https://github.com/apache/hudi/issues/7314#issuecomment-1330031626
> Yeah, seems a bug, seems all the invocations of `#writeTableMetadata` are already guarded by the txn lock. We can remove the lock in `#writeTableMetadata` i think, please fire a JIRA and fix it if you are interested.
ok, I'll try
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] aBlackAnt commented on issue #7314: [SUPPORT] Caused by: java.lang.IllegalArgumentException: ALREADY_ACQUIRED
Posted by GitBox <gi...@apache.org>.
aBlackAnt commented on issue #7314:
URL: https://github.com/apache/hudi/issues/7314#issuecomment-1328779002
I think that in ``BaseHoodieWriteClient.java``, ``this.txnManager.beginTransaction`` and ``commit`` both try get lock. Is that right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on issue #7314: [SUPPORT] Caused by: java.lang.IllegalArgumentException: ALREADY_ACQUIRED
Posted by GitBox <gi...@apache.org>.
danny0405 commented on issue #7314:
URL: https://github.com/apache/hudi/issues/7314#issuecomment-1330023180
Yeah, seems a bug, we need some flag to indicate whether the lock is required.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] just-JL commented on issue #7314: [SUPPORT] Caused by: java.lang.IllegalArgumentException: ALREADY_ACQUIRED
Posted by GitBox <gi...@apache.org>.
just-JL commented on issue #7314:
URL: https://github.com/apache/hudi/issues/7314#issuecomment-1330085896
@danny0405 Please take a look at this pr #7320. If there is a problem, I will update.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] KnightChess commented on issue #7314: [SUPPORT] Caused by: java.lang.IllegalArgumentException: ALREADY_ACQUIRED
Posted by GitBox <gi...@apache.org>.
KnightChess commented on issue #7314:
URL: https://github.com/apache/hudi/issues/7314#issuecomment-1329245041
is https://github.com/apache/hudi/pull/6272 can resolve you problem?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan closed issue #7314: [SUPPORT] Caused by: java.lang.IllegalArgumentException: ALREADY_ACQUIRED
Posted by GitBox <gi...@apache.org>.
nsivabalan closed issue #7314: [SUPPORT] Caused by: java.lang.IllegalArgumentException: ALREADY_ACQUIRED
URL: https://github.com/apache/hudi/issues/7314
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] KnightChess commented on issue #7314: [SUPPORT] Caused by: java.lang.IllegalArgumentException: ALREADY_ACQUIRED
Posted by GitBox <gi...@apache.org>.
KnightChess commented on issue #7314:
URL: https://github.com/apache/hudi/issues/7314#issuecomment-1330005642
if flink support occ, I think it is a bug @danny0405 . And the lock hudi support I think is not a reentrant lock
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] just-JL closed issue #7314: [SUPPORT] Caused by: java.lang.IllegalArgumentException: ALREADY_ACQUIRED
Posted by GitBox <gi...@apache.org>.
just-JL closed issue #7314: [SUPPORT] Caused by: java.lang.IllegalArgumentException: ALREADY_ACQUIRED
URL: https://github.com/apache/hudi/issues/7314
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on issue #7314: [SUPPORT] Caused by: java.lang.IllegalArgumentException: ALREADY_ACQUIRED
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on issue #7314:
URL: https://github.com/apache/hudi/issues/7314#issuecomment-1340307253
thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org