You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Y Ethan Guo (Jira)" <ji...@apache.org> on 2024/09/23 18:09:00 UTC

[jira] [Updated] (HUDI-8090) New zookeeper based lock provider

     [ https://issues.apache.org/jira/browse/HUDI-8090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Y Ethan Guo updated HUDI-8090:
------------------------------
    Reviewers: Y Ethan Guo

> New zookeeper based lock provider
> ---------------------------------
>
>                 Key: HUDI-8090
>                 URL: https://issues.apache.org/jira/browse/HUDI-8090
>             Project: Apache Hudi
>          Issue Type: Improvement
>            Reporter: Davis Zhang
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 1.0.0
>
>
> Add a new lock provider which is exactly the same as existing zookeeper based one except the way the partition key part is handled. To be specific:
>  * It requiresTable name as indicated by "hoodie.table.name".
>  * It requiresTable base path "hoodie.base.path".
>  * The zookeeper base path key ([hoodie.write.lock.dynamodb.partition_key|https://hudi.apache.org/docs/configurations/#hoodiewritelockdynamodbpartition_key]) is automatically derived as /tmp/{{{}<table name>-<hash of the table base path>{}}}. To check the exact value, please check relevant log from class 
> org.apache.hudi.client.transaction.lock.ZookeeperBasedImplicitBasePathLockProvider.
>  * It will ignore value set by`
> hoodie.write.lock.zookeeper.base_path` and `hoodie.write.lock.zookeeper.lock_key`.
> Test:
> Unit test
> h3. Change Logs
> Add new lock provider implementation. For code shared by the new-old lock provider, they are extracted to a base class.
> Unit test + integration test
> h3. Impact
> None.
> h3. Risk level (write none, low medium or high below)
> no risk as of now since no one is using the new lock provider.
> h3. Documentation Update
> In [https://hudi.apache.org/docs/concurrency_control/], we need to add description for the newly added lock provider.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)