You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by yi...@apache.org on 2023/01/13 23:14:22 UTC
[hudi] branch master updated: [MINOR] Fix minor issues in HoodieMetadataTableValidator docs (#7518)
This is an automated email from the ASF dual-hosted git repository.
yihua pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/master by this push:
new fcc508c8c0a [MINOR] Fix minor issues in HoodieMetadataTableValidator docs (#7518)
fcc508c8c0a is described below
commit fcc508c8c0a549a1e14ff1dcb2a66c30c99ad421
Author: Y Ethan Guo <et...@gmail.com>
AuthorDate: Fri Jan 13 15:14:12 2023 -0800
[MINOR] Fix minor issues in HoodieMetadataTableValidator docs (#7518)
---
.../utilities/HoodieMetadataTableValidator.java | 49 +++++++++++-----------
1 file changed, 24 insertions(+), 25 deletions(-)
diff --git a/hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieMetadataTableValidator.java b/hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieMetadataTableValidator.java
index 0b3c072c92a..77e39af38ff 100644
--- a/hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieMetadataTableValidator.java
+++ b/hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieMetadataTableValidator.java
@@ -99,12 +99,12 @@ import static org.apache.hudi.hadoop.CachingPath.getPathWithoutSchemeAndAuthorit
* between metadata table and filesystem.
* <p>
* There are five validation tasks, that can be enabled independently through the following CLI options:
- * - `--validate-latest-file-slices`: validate latest file slices for all partitions.
- * - `--validate-latest-base-files`: validate latest base files for all partitions.
+ * - `--validate-latest-file-slices`: validate the latest file slices for all partitions.
+ * - `--validate-latest-base-files`: validate the latest base files for all partitions.
* - `--validate-all-file-groups`: validate all file groups, and all file slices within file groups.
* - `--validate-all-column-stats`: validate column stats for all columns in the schema
* - `--validate-bloom-filters`: validate bloom filters of base files
- *
+ * <p>
* If the Hudi table is on the local file system, the base path passed to `--base-path` must have
* "file:" prefix to avoid validation failure.
* <p>
@@ -113,37 +113,36 @@ import static org.apache.hudi.hadoop.CachingPath.getPathWithoutSchemeAndAuthorit
* Example command:
* ```
* spark-submit \
- * --class org.apache.hudi.utilities.HoodieMetadataTableValidator \
- * --master spark://xxxx:7077 \
- * --driver-memory 1g \
- * --executor-memory 1g \
- * $HUDI_DIR/hudi/packaging/hudi-utilities-bundle/target/hudi-utilities-bundle_2.11-0.11.0-SNAPSHOT.jar \
- * --base-path basePath \
- * --validate-latest-file-slices \
- * --validate-latest-base-files \
- * --validate-all-file-groups
+ * --class org.apache.hudi.utilities.HoodieMetadataTableValidator \
+ * --master spark://xxxx:7077 \
+ * --driver-memory 1g \
+ * --executor-memory 1g \
+ * $HUDI_DIR/packaging/hudi-utilities-bundle/target/hudi-utilities-bundle_2.11-0.13.0-SNAPSHOT.jar \
+ * --base-path basePath \
+ * --validate-latest-file-slices \
+ * --validate-latest-base-files \
+ * --validate-all-file-groups
* ```
*
* <p>
- * Also You can set `--continuous` for long running this validator.
+ * Also, You can set `--continuous` for long running this validator.
* And use `--min-validate-interval-seconds` to control the validation frequency, default is 10 minutes.
* <p>
* Example command:
* ```
* spark-submit \
- * --class org.apache.hudi.utilities.HoodieMetadataTableValidator \
- * --master spark://xxxx:7077 \
- * --driver-memory 1g \
- * --executor-memory 1g \
- * $HUDI_DIR/hudi/packaging/hudi-utilities-bundle/target/hudi-utilities-bundle_2.11-0.11.0-SNAPSHOT.jar \
- * --base-path basePath \
- * --validate-latest-file-slices \
- * --validate-latest-base-files \
- * --validate-all-file-groups \
- * --continuous \
- * --min-validate-interval-seconds 60
+ * --class org.apache.hudi.utilities.HoodieMetadataTableValidator \
+ * --master spark://xxxx:7077 \
+ * --driver-memory 1g \
+ * --executor-memory 1g \
+ * $HUDI_DIR/packaging/hudi-utilities-bundle/target/hudi-utilities-bundle_2.11-0.13.0-SNAPSHOT.jar \
+ * --base-path basePath \
+ * --validate-latest-file-slices \
+ * --validate-latest-base-files \
+ * --validate-all-file-groups \
+ * --continuous \
+ * --min-validate-interval-seconds 60
* ```
- *
*/
public class HoodieMetadataTableValidator implements Serializable {