You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "HuangZhenQiu (via GitHub)" <gi...@apache.org> on 2026/08/27 06:05:10 UTC
[PR] feat(flink): add new RocksDBPartitionedIndexBackend [hudi]
HuangZhenQiu opened a new pull request, #19765:
URL: https://github.com/apache/hudi/pull/19765
### Describe the issue this Pull Request addresses
add new RocksDBPartitionedIndexBackend that can manages record-key-to-fileId mapping for each partition in column family of RocksDB.
### Summary and Changelog
1. Add RocksDBPartitionedIndexBackend, a RocksDB-based implementation of PartitionedIndexBackend that stores each data partition's record-key-to-fileId mapping in its own column family, enabling per-partition eviction.
2. Add PartitionedIndexBackendFactory to centralize selection between the dummy, record-level, and new RocksDB-backed index backends based on FlinkOptions.INDEX_RLI_BACKEND_TYPE.
3. Update DynamicBucketAssignFunction to use the new factory instead of inlining backend construction.
4. Add unit tests for the new factory and RocksDB backend.
### Impact
none
### Risk Level
none
### Documentation Update
none
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Enough context is provided in the sections above
- [ ] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): add new RocksDBPartitionedIndexBackend [hudi]
Posted by "HuangZhenQiu (via GitHub)" <gi...@apache.org>.
HuangZhenQiu commented on code in PR #19765:
URL: https://github.com/apache/hudi/pull/19765#discussion_r3881608372
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/partitioner/index/PartitionedIndexBackendFactory.java:
##########
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.sink.partitioner.index;
+
+import org.apache.hudi.configuration.FlinkOptions;
+
+import org.apache.flink.configuration.Configuration;
+
+/**
+ * Factory to create a {@link PartitionedIndexBackend} used by the dynamic bucket assign function.
+ */
+public class PartitionedIndexBackendFactory {
+ private static final String ROCKSDB_BACKEND_TYPE = "rocksdb";
+
+ /**
+ * Creates the partitioned index backend used to look up and record {@code recordKey -> fileGroupId}
+ * mappings for the partitioned record level index.
+ *
+ * @param conf Flink write configuration
+ * @param isInsertOverwrite whether the write operation is an insert overwrite, in which case indexing is skipped
+ * @param bootstrapFilter filter for deciding whether a bootstrapped RLI record belongs to this task,
+ * used only by the metadata-table-backed backend
+ * @return partitioned index backend for record-key lookups scoped to a data partition
+ */
+ public static PartitionedIndexBackend create(
+ Configuration conf,
+ boolean isInsertOverwrite,
+ RecordLevelIndexBackend.BootstrapFilter bootstrapFilter) {
+ if (isInsertOverwrite) {
+ return new DummyPartitionedIndexBackend();
+ }
+ String backendType = conf.get(FlinkOptions.INDEX_RLI_BACKEND_TYPE);
+ if (ROCKSDB_BACKEND_TYPE.equalsIgnoreCase(backendType)) {
Review Comment:
Thanks for calling it out. Make it unelectable now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): add new RocksDBPartitionedIndexBackend [hudi]
Posted by "HuangZhenQiu (via GitHub)" <gi...@apache.org>.
HuangZhenQiu commented on PR #19765:
URL: https://github.com/apache/hudi/pull/19765#issuecomment-5480463920
It is a good question. As we described in the RFC, we probably need to decouple the dynamic partitioned record index load and final lookup into two operators. When a record of old partition comes, it will trigger the partition record index load, these record index should be keyed by and shuttle to all of other task managers. In this way, the RocksDB in each of task managers only needs to store the state it required, and one partition need to load once.
The RocksDBPartitionedIndexBackend will be a configurable backend owned by BucketAssignFunction and the new function PreLookup. The RocksDB Dao will be initialized in singleton and shared by both BucketAssignFunction and PreLookup. How do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): add new RocksDBPartitionedIndexBackend [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19765:
URL: https://github.com/apache/hudi/pull/19765#issuecomment-5454308805
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "836d4e6367586c05914f11803a34c5b7b3c4f29f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16613",
"triggerID" : "836d4e6367586c05914f11803a34c5b7b3c4f29f",
"triggerType" : "PUSH"
}, {
"hash" : "5d06f520af520f0785504918eb01bdf7c56e970b",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16615",
"triggerID" : "5d06f520af520f0785504918eb01bdf7c56e970b",
"triggerType" : "PUSH"
}, {
"hash" : "5c2cafe8f0d756cc157ad930b2ba0ca322da38a1",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16678",
"triggerID" : "5c2cafe8f0d756cc157ad930b2ba0ca322da38a1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5d06f520af520f0785504918eb01bdf7c56e970b Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16615)
* 5c2cafe8f0d756cc157ad930b2ba0ca322da38a1 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16678)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): add new RocksDBPartitionedIndexBackend [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19765:
URL: https://github.com/apache/hudi/pull/19765#issuecomment-5455236013
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "836d4e6367586c05914f11803a34c5b7b3c4f29f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16613",
"triggerID" : "836d4e6367586c05914f11803a34c5b7b3c4f29f",
"triggerType" : "PUSH"
}, {
"hash" : "5d06f520af520f0785504918eb01bdf7c56e970b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16615",
"triggerID" : "5d06f520af520f0785504918eb01bdf7c56e970b",
"triggerType" : "PUSH"
}, {
"hash" : "5c2cafe8f0d756cc157ad930b2ba0ca322da38a1",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16678",
"triggerID" : "5c2cafe8f0d756cc157ad930b2ba0ca322da38a1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5c2cafe8f0d756cc157ad930b2ba0ca322da38a1 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16678)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): add new RocksDBPartitionedIndexBackend [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19765:
URL: https://github.com/apache/hudi/pull/19765#issuecomment-5435172077
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "836d4e6367586c05914f11803a34c5b7b3c4f29f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16613",
"triggerID" : "836d4e6367586c05914f11803a34c5b7b3c4f29f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 836d4e6367586c05914f11803a34c5b7b3c4f29f Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16613)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): add new RocksDBPartitionedIndexBackend [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #19765:
URL: https://github.com/apache/hudi/pull/19765#issuecomment-5435173956
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/19765?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 77.97%. Comparing base ([`18ae8c3`](https://app.codecov.io/gh/apache/hudi/commit/18ae8c349058140f25a4f7247ff4a4bc8fc56dec?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`836d4e6`](https://app.codecov.io/gh/apache/hudi/commit/836d4e6367586c05914f11803a34c5b7b3c4f29f?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #19765 +/- ##
=========================================
Coverage 77.97% 77.97%
+ Complexity 33464 33461 -3
=========================================
Files 2539 2539
Lines 140974 140971 -3
Branches 17013 17013
=========================================
- Hits 109928 109927 -1
+ Misses 23382 23381 -1
+ Partials 7664 7663 -1
```
| [Components](https://app.codecov.io/gh/apache/hudi/pull/19765/components?src=pr&el=components&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hudi-common](https://app.codecov.io/gh/apache/hudi/pull/19765/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `83.49% <ø> (ø)` | |
| [hudi-client](https://app.codecov.io/gh/apache/hudi/pull/19765/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `83.08% <ø> (+<0.01%)` | :arrow_up: |
| [hudi-flink](https://app.codecov.io/gh/apache/hudi/pull/19765/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `85.62% <ø> (-0.01%)` | :arrow_down: |
| [hudi-spark-datasource](https://app.codecov.io/gh/apache/hudi/pull/19765/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `72.39% <ø> (ø)` | |
| [hudi-utilities](https://app.codecov.io/gh/apache/hudi/pull/19765/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `74.32% <ø> (ø)` | |
| [hudi-cli](https://app.codecov.io/gh/apache/hudi/pull/19765/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `15.06% <ø> (ø)` | |
| [hudi-hadoop](https://app.codecov.io/gh/apache/hudi/pull/19765/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `69.24% <ø> (ø)` | |
| [hudi-sync](https://app.codecov.io/gh/apache/hudi/pull/19765/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `75.54% <ø> (ø)` | |
| [hudi-io](https://app.codecov.io/gh/apache/hudi/pull/19765/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `79.85% <ø> (ø)` | |
| [hudi-timeline-service](https://app.codecov.io/gh/apache/hudi/pull/19765/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `83.74% <ø> (ø)` | |
| [hudi-cloud](https://app.codecov.io/gh/apache/hudi/pull/19765/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `64.27% <ø> (ø)` | |
| [hudi-kafka-connect](https://app.codecov.io/gh/apache/hudi/pull/19765/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `53.20% <ø> (ø)` | |
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/19765/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/19765/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `43.69% <ø> (-0.04%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/19765?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [.../sink/partitioner/DynamicBucketAssignFunction.java](https://app.codecov.io/gh/apache/hudi/pull/19765?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fsink%2Fpartitioner%2FDynamicBucketAssignFunction.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvc2luay9wYXJ0aXRpb25lci9EeW5hbWljQnVja2V0QXNzaWduRnVuY3Rpb24uamF2YQ==) | `97.77% <ø> (-0.14%)` | :arrow_down: |
... and [1 file with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/19765/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): add new RocksDBPartitionedIndexBackend [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19765:
URL: https://github.com/apache/hudi/pull/19765#issuecomment-5454142519
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "836d4e6367586c05914f11803a34c5b7b3c4f29f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16613",
"triggerID" : "836d4e6367586c05914f11803a34c5b7b3c4f29f",
"triggerType" : "PUSH"
}, {
"hash" : "5d06f520af520f0785504918eb01bdf7c56e970b",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16615",
"triggerID" : "5d06f520af520f0785504918eb01bdf7c56e970b",
"triggerType" : "PUSH"
}, {
"hash" : "5c2cafe8f0d756cc157ad930b2ba0ca322da38a1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5c2cafe8f0d756cc157ad930b2ba0ca322da38a1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5d06f520af520f0785504918eb01bdf7c56e970b Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16615)
* 5c2cafe8f0d756cc157ad930b2ba0ca322da38a1 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): add new RocksDBPartitionedIndexBackend [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19765:
URL: https://github.com/apache/hudi/pull/19765#issuecomment-5435690297
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "836d4e6367586c05914f11803a34c5b7b3c4f29f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16613",
"triggerID" : "836d4e6367586c05914f11803a34c5b7b3c4f29f",
"triggerType" : "PUSH"
}, {
"hash" : "5d06f520af520f0785504918eb01bdf7c56e970b",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16615",
"triggerID" : "5d06f520af520f0785504918eb01bdf7c56e970b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 836d4e6367586c05914f11803a34c5b7b3c4f29f Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16613)
* 5d06f520af520f0785504918eb01bdf7c56e970b Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16615)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): add new RocksDBPartitionedIndexBackend [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19765:
URL: https://github.com/apache/hudi/pull/19765#issuecomment-5436698254
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "836d4e6367586c05914f11803a34c5b7b3c4f29f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16613",
"triggerID" : "836d4e6367586c05914f11803a34c5b7b3c4f29f",
"triggerType" : "PUSH"
}, {
"hash" : "5d06f520af520f0785504918eb01bdf7c56e970b",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16615",
"triggerID" : "5d06f520af520f0785504918eb01bdf7c56e970b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5d06f520af520f0785504918eb01bdf7c56e970b Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16615)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): add new RocksDBPartitionedIndexBackend [hudi]
Posted by "hudi-agent (via GitHub)" <gi...@apache.org>.
hudi-agent commented on code in PR #19765:
URL: https://github.com/apache/hudi/pull/19765#discussion_r3869195302
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/partitioner/index/PartitionedIndexBackendFactory.java:
##########
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.sink.partitioner.index;
+
+import org.apache.hudi.configuration.FlinkOptions;
+
+import org.apache.flink.configuration.Configuration;
+
+/**
+ * Factory to create a {@link PartitionedIndexBackend} used by the dynamic bucket assign function.
+ */
+public class PartitionedIndexBackendFactory {
+ private static final String ROCKSDB_BACKEND_TYPE = "rocksdb";
+
+ /**
+ * Creates the partitioned index backend used to look up and record {@code recordKey -> fileGroupId}
+ * mappings for the partitioned record level index.
+ *
+ * @param conf Flink write configuration
+ * @param isInsertOverwrite whether the write operation is an insert overwrite, in which case indexing is skipped
+ * @param bootstrapFilter filter for deciding whether a bootstrapped RLI record belongs to this task,
+ * used only by the metadata-table-backed backend
+ * @return partitioned index backend for record-key lookups scoped to a data partition
+ */
+ public static PartitionedIndexBackend create(
+ Configuration conf,
+ boolean isInsertOverwrite,
+ RecordLevelIndexBackend.BootstrapFilter bootstrapFilter) {
+ if (isInsertOverwrite) {
+ return new DummyPartitionedIndexBackend();
+ }
+ String backendType = conf.get(FlinkOptions.INDEX_RLI_BACKEND_TYPE);
+ if (ROCKSDB_BACKEND_TYPE.equalsIgnoreCase(backendType)) {
+ return new RocksDBPartitionedIndexBackend(conf.get(FlinkOptions.INDEX_RLI_CACHE_ROCKSDB_BASE_PATH));
Review Comment:
🤖 With `index.rli.backend.type=rocksdb`, `get()` only sees keys written via `update()` in the current run (RocksDBDAO wipes its dir on startup) and never consults the MDT RLI. On a non-empty table or after a restart, existing keys miss and get routed as inserts in DynamicBucketAssignFunction (L130-136), which can create a second file group for a key that already exists. The `index.rli.cache.rocksdb.base.path` description says this cache sits "in front of the metadata table" — is the MDT bootstrap/fallback still TODO before this backend is selectable? @danny0405 might want to weigh in.
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag quality.</i></sub>
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/partitioner/index/PartitionedIndexBackendFactory.java:
##########
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.sink.partitioner.index;
+
+import org.apache.hudi.configuration.FlinkOptions;
+
+import org.apache.flink.configuration.Configuration;
+
+/**
+ * Factory to create a {@link PartitionedIndexBackend} used by the dynamic bucket assign function.
+ */
+public class PartitionedIndexBackendFactory {
+ private static final String ROCKSDB_BACKEND_TYPE = "rocksdb";
+
+ /**
Review Comment:
🤖 nit: the string `"rocksdb"` is defined here but the valid values for `FlinkOptions.INDEX_RLI_BACKEND_TYPE` aren't visible at the call site. It might be clearer to keep this constant (or an enum) in `FlinkOptions` alongside the config key itself, so the two can't drift independently.
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag quality.</i></sub>
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/partitioner/index/RocksDBPartitionedIndexBackend.java:
##########
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.sink.partitioner.index;
+
+import org.apache.hudi.common.util.StringUtils;
+import org.apache.hudi.common.util.collection.Pair;
+import org.apache.hudi.common.util.collection.RocksDBDAO;
+
+import lombok.extern.slf4j.Slf4j;
+import org.rocksdb.RocksDB;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.stream.Collectors;
+
+/**
+ * An implementation of {@link PartitionedIndexBackend} based on RocksDB.
+ *
+ * <p>Each data partition is stored in its own RocksDB column family, so that a partition's mapping
+ * can be dropped as a unit. A partition is only registered as visible to {@link #get} in RocksDB's
+ * own {@code default} column family after its own column family has been created, so a lookup never
+ * sees a partition whose column family creation is still in flight.
+ *
+ * <p>This backend operates against an already-open RocksDB instance: it does not bootstrap partitions
+ * from the metadata table and does not implement TTL-based eviction of partitions.
+ */
+@Slf4j
+public class RocksDBPartitionedIndexBackend implements PartitionedIndexBackend {
+ private static final String PARTITION_COLUMN_FAMILY_PREFIX = "pcf_";
+ private static final String REGISTRY_COLUMN_FAMILY = StringUtils.fromUTF8Bytes(RocksDB.DEFAULT_COLUMN_FAMILY);
+
+ private final RocksDBDAO rocksDBDAO;
+
+ public RocksDBPartitionedIndexBackend(String rocksDbBasePath) {
+ this.rocksDBDAO = new RocksDBDAO("hudi-partitioned-index-backend", rocksDbBasePath, new ConcurrentHashMap<>(), true);
+ }
+
+ @Override
+ public String get(String partitionPath, String recordKey) {
+ if (!isPartitionRegistered(partitionPath)) {
+ return null;
+ }
+ return this.rocksDBDAO.get(partitionColumnFamily(partitionPath), recordKey);
+ }
+
+ @Override
+ public void update(String partitionPath, String recordKey, String fileId) {
+ String columnFamily = partitionColumnFamily(partitionPath);
+ if (!this.rocksDBDAO.columnFamilyExists(columnFamily)) {
+ this.rocksDBDAO.addColumnFamily(columnFamily);
+ }
+ this.rocksDBDAO.put(columnFamily, recordKey, fileId);
+ if (!isPartitionRegistered(partitionPath)) {
+ registerPartition(partitionPath);
+ }
+ }
+
+ /**
+ * Returns whether the given partition's column family has been fully created and is safe to read from.
+ *
+ * @param partitionPath the partition path to check
+ */
+ public boolean isPartitionRegistered(String partitionPath) {
+ Boolean registered = this.rocksDBDAO.get(REGISTRY_COLUMN_FAMILY, partitionPath);
+ return registered != null && registered;
+ }
+
+ /**
+ * Lists the partitions currently registered in this backend.
+ */
+ public List<String> listRegisteredPartitions() {
+ return this.rocksDBDAO.<Boolean>prefixSearch(REGISTRY_COLUMN_FAMILY, "")
+ .map(Pair::getKey)
+ .collect(Collectors.toList());
+ }
+
+ /**
+ * Drops a partition's column family and removes it from the registry.
+ *
+ * @param partitionPath the partition path to delete
+ */
+ public void deletePartition(String partitionPath) {
Review Comment:
🤖 `deletePartition` drops the column family before removing the registry entry — the reverse of the register-after-create ordering `update()` uses to keep `get()` safe. If a `get()` ever observed the intermediate state it would see the partition still registered but the CF handle already removed from RocksDBDAO's managedHandlesMap (null handle passed to RocksDB.get). Harmless while the assign operator is single-threaded, but would it be safer to unregister first, then drop the CF, to match the documented visibility invariant?
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag quality.</i></sub>
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/partitioner/index/RocksDBPartitionedIndexBackend.java:
##########
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.sink.partitioner.index;
+
+import org.apache.hudi.common.util.StringUtils;
+import org.apache.hudi.common.util.collection.Pair;
+import org.apache.hudi.common.util.collection.RocksDBDAO;
+
+import lombok.extern.slf4j.Slf4j;
+import org.rocksdb.RocksDB;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.stream.Collectors;
+
+/**
+ * An implementation of {@link PartitionedIndexBackend} based on RocksDB.
+ *
+ * <p>Each data partition is stored in its own RocksDB column family, so that a partition's mapping
+ * can be dropped as a unit. A partition is only registered as visible to {@link #get} in RocksDB's
+ * own {@code default} column family after its own column family has been created, so a lookup never
+ * sees a partition whose column family creation is still in flight.
+ *
+ * <p>This backend operates against an already-open RocksDB instance: it does not bootstrap partitions
+ * from the metadata table and does not implement TTL-based eviction of partitions.
+ */
+@Slf4j
+public class RocksDBPartitionedIndexBackend implements PartitionedIndexBackend {
+ private static final String PARTITION_COLUMN_FAMILY_PREFIX = "pcf_";
+ private static final String REGISTRY_COLUMN_FAMILY = StringUtils.fromUTF8Bytes(RocksDB.DEFAULT_COLUMN_FAMILY);
+
+ private final RocksDBDAO rocksDBDAO;
+
+ public RocksDBPartitionedIndexBackend(String rocksDbBasePath) {
+ this.rocksDBDAO = new RocksDBDAO("hudi-partitioned-index-backend", rocksDbBasePath, new ConcurrentHashMap<>(), true);
+ }
+
+ @Override
Review Comment:
🤖 nit: the parameter `rocksDbBasePath` uses lowercase `Db` while the class name spells it `RocksDB` (uppercase). Could you align to `rocksDBBasePath` for consistency?
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): add new RocksDBPartitionedIndexBackend [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19765:
URL: https://github.com/apache/hudi/pull/19765#issuecomment-5435127496
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "836d4e6367586c05914f11803a34c5b7b3c4f29f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "836d4e6367586c05914f11803a34c5b7b3c4f29f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 836d4e6367586c05914f11803a34c5b7b3c4f29f UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): add new RocksDBPartitionedIndexBackend [hudi]
Posted by "cshuo (via GitHub)" <gi...@apache.org>.
cshuo commented on PR #19765:
URL: https://github.com/apache/hudi/pull/19765#issuecomment-5473456932
Is `RocksDBPartitionedIndexBackend` intended to be a complete backend owned by `BucketAssigner`, or only a low-level store?
Currently, the `RecordLevelIndexBackend` owns bootstrap and cleanup itself. Will the RocksDB bootstrap and cleanup logic also live in `BucketAssigner`, or in a separate component such as `RLIBootstrapOperator`? If external, how will it share the backend’s privately owned RocksDB instance? Could you clarify the final ownership and responsibility boundary?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): add new RocksDBPartitionedIndexBackend [hudi]
Posted by "cshuo (via GitHub)" <gi...@apache.org>.
cshuo commented on code in PR #19765:
URL: https://github.com/apache/hudi/pull/19765#discussion_r3877855524
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/partitioner/index/PartitionedIndexBackendFactory.java:
##########
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.sink.partitioner.index;
+
+import org.apache.hudi.configuration.FlinkOptions;
+
+import org.apache.flink.configuration.Configuration;
+
+/**
+ * Factory to create a {@link PartitionedIndexBackend} used by the dynamic bucket assign function.
+ */
+public class PartitionedIndexBackendFactory {
+ private static final String ROCKSDB_BACKEND_TYPE = "rocksdb";
+
+ /**
+ * Creates the partitioned index backend used to look up and record {@code recordKey -> fileGroupId}
+ * mappings for the partitioned record level index.
+ *
+ * @param conf Flink write configuration
+ * @param isInsertOverwrite whether the write operation is an insert overwrite, in which case indexing is skipped
+ * @param bootstrapFilter filter for deciding whether a bootstrapped RLI record belongs to this task,
+ * used only by the metadata-table-backed backend
+ * @return partitioned index backend for record-key lookups scoped to a data partition
+ */
+ public static PartitionedIndexBackend create(
+ Configuration conf,
+ boolean isInsertOverwrite,
+ RecordLevelIndexBackend.BootstrapFilter bootstrapFilter) {
+ if (isInsertOverwrite) {
+ return new DummyPartitionedIndexBackend();
+ }
+ String backendType = conf.get(FlinkOptions.INDEX_RLI_BACKEND_TYPE);
+ if (ROCKSDB_BACKEND_TYPE.equalsIgnoreCase(backendType)) {
Review Comment:
Selecting `rocksdb` here bypasses the MDT-backed `RecordLevelIndexBackend`, but this backend starts empty and neither bootstraps from nor falls back to the authoritative MDT RLI. On a non-empty table or after task/job restart, committed keys therefore miss in `get()` and `DynamicBucketAssignFunction` routes them as inserts, potentially assigning a second file group to an existing key.
In addition, the first `update()` registers the column family after loading only one key, which violates RFC-107's partition-completeness invariant. Please keep this backend non-selectable until partition bootstrap/on-demand MDT loading is implemented, or compose it with the MDT backend and register a partition only after its complete RLI state has been loaded.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): add new RocksDBPartitionedIndexBackend [hudi]
Posted by "HuangZhenQiu (via GitHub)" <gi...@apache.org>.
HuangZhenQiu commented on code in PR #19765:
URL: https://github.com/apache/hudi/pull/19765#discussion_r3881608372
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/partitioner/index/PartitionedIndexBackendFactory.java:
##########
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.sink.partitioner.index;
+
+import org.apache.hudi.configuration.FlinkOptions;
+
+import org.apache.flink.configuration.Configuration;
+
+/**
+ * Factory to create a {@link PartitionedIndexBackend} used by the dynamic bucket assign function.
+ */
+public class PartitionedIndexBackendFactory {
+ private static final String ROCKSDB_BACKEND_TYPE = "rocksdb";
+
+ /**
+ * Creates the partitioned index backend used to look up and record {@code recordKey -> fileGroupId}
+ * mappings for the partitioned record level index.
+ *
+ * @param conf Flink write configuration
+ * @param isInsertOverwrite whether the write operation is an insert overwrite, in which case indexing is skipped
+ * @param bootstrapFilter filter for deciding whether a bootstrapped RLI record belongs to this task,
+ * used only by the metadata-table-backed backend
+ * @return partitioned index backend for record-key lookups scoped to a data partition
+ */
+ public static PartitionedIndexBackend create(
+ Configuration conf,
+ boolean isInsertOverwrite,
+ RecordLevelIndexBackend.BootstrapFilter bootstrapFilter) {
+ if (isInsertOverwrite) {
+ return new DummyPartitionedIndexBackend();
+ }
+ String backendType = conf.get(FlinkOptions.INDEX_RLI_BACKEND_TYPE);
+ if (ROCKSDB_BACKEND_TYPE.equalsIgnoreCase(backendType)) {
Review Comment:
Thanks for calling it out. Make the rocksdb option unselectable now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): add new RocksDBPartitionedIndexBackend [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19765:
URL: https://github.com/apache/hudi/pull/19765#issuecomment-5436099963
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "836d4e6367586c05914f11803a34c5b7b3c4f29f",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16613",
"triggerID" : "836d4e6367586c05914f11803a34c5b7b3c4f29f",
"triggerType" : "PUSH"
}, {
"hash" : "5d06f520af520f0785504918eb01bdf7c56e970b",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16615",
"triggerID" : "5d06f520af520f0785504918eb01bdf7c56e970b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 836d4e6367586c05914f11803a34c5b7b3c4f29f Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16613)
* 5d06f520af520f0785504918eb01bdf7c56e970b Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16615)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): add new RocksDBPartitionedIndexBackend [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19765:
URL: https://github.com/apache/hudi/pull/19765#issuecomment-5435581111
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "836d4e6367586c05914f11803a34c5b7b3c4f29f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16613",
"triggerID" : "836d4e6367586c05914f11803a34c5b7b3c4f29f",
"triggerType" : "PUSH"
}, {
"hash" : "5d06f520af520f0785504918eb01bdf7c56e970b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5d06f520af520f0785504918eb01bdf7c56e970b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 836d4e6367586c05914f11803a34c5b7b3c4f29f Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16613)
* 5d06f520af520f0785504918eb01bdf7c56e970b UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org