You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Y Ethan Guo (Jira)" <ji...@apache.org> on 2024/09/23 18:09:00 UTC
[jira] [Updated] (HUDI-8090) New zookeeper based lock provider
[ https://issues.apache.org/jira/browse/HUDI-8090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Y Ethan Guo updated HUDI-8090:
------------------------------
Reviewers: Y Ethan Guo
> New zookeeper based lock provider
> ---------------------------------
>
> Key: HUDI-8090
> URL: https://issues.apache.org/jira/browse/HUDI-8090
> Project: Apache Hudi
> Issue Type: Improvement
> Reporter: Davis Zhang
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.0.0
>
>
> Add a new lock provider which is exactly the same as existing zookeeper based one except the way the partition key part is handled. To be specific:
> * It requiresTable name as indicated by "hoodie.table.name".
> * It requiresTable base path "hoodie.base.path".
> * The zookeeper base path key ([hoodie.write.lock.dynamodb.partition_key|https://hudi.apache.org/docs/configurations/#hoodiewritelockdynamodbpartition_key]) is automatically derived as /tmp/{{{}<table name>-<hash of the table base path>{}}}. To check the exact value, please check relevant log from classÂ
> org.apache.hudi.client.transaction.lock.ZookeeperBasedImplicitBasePathLockProvider.
> * It will ignore value set by`
> hoodie.write.lock.zookeeper.base_path` and `hoodie.write.lock.zookeeper.lock_key`.
> Test:
> Unit test
> h3. Change Logs
> Add new lock provider implementation. For code shared by the new-old lock provider, they are extracted to a base class.
> Unit test + integration test
> h3. Impact
> None.
> h3. Risk level (write none, low medium or high below)
> no risk as of now since no one is using the new lock provider.
> h3. Documentation Update
> In [https://hudi.apache.org/docs/concurrency_control/], we need to add description for the newly added lock provider.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)