You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "yihua (via GitHub)" <gi...@apache.org> on 2023/03/14 07:33:42 UTC
[GitHub] [hudi] yihua opened a new pull request, #8176: [HUDI-5929] Automatically infer key generator type
yihua opened a new pull request, #8176:
URL: https://github.com/apache/hudi/pull/8176
### Change Logs
This PR applies the logic of automatically inferring the key generator to Deltastreamer, besides Spark Datasource, Streaming and SQL writers.
- Moves the logic of automatic inference of key generator to `KeyGenUtils`
- Refactors `HoodieSparkKeyGeneratorFactory` for code reuse
- Fixes `DeltaSync` for using the inferred key generator
- Removes unnecessary key generator configs in a few tests
- Adds new tests in `TestHoodieDeltaStreamer`, `TestKeyGenUtils`, and `TestHoodieSparkKeyGeneratorFactory` to verify the automatic inference of the key generator type and class name.
Note that, if either `hoodie.datasource.write.keygenerator.class` or `hoodie.datasource.write.keygenerator.type` is explicitly set, the configured value takes precedence to be in effect (`hoodie.datasource.write.keygenerator.class` first, and then `hoodie.datasource.write.keygenerator.type`) and the inference of the key generator is not triggered.
### Impact
In most of cases (for key generator types of SIMPLE, COMPLEX, and NON_PARTITION), user does not need to explicitly specify the key generator type anymore. The key generator is automatically inferred.
### Risk level
low
### Documentation Update
Need to update the documentation around code examples where key generator configs are not mandatory.
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8176: [HUDI-5929] Automatically infer key generator type
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8176:
URL: https://github.com/apache/hudi/pull/8176#issuecomment-1492251676
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8ac30b3eed9452b6c7ed2715748dd9866770961f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16002",
"triggerID" : "8ac30b3eed9452b6c7ed2715748dd9866770961f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8ac30b3eed9452b6c7ed2715748dd9866770961f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16002)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] yihua merged pull request #8176: [HUDI-5929] Automatically infer key generator type
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua merged PR #8176:
URL: https://github.com/apache/hudi/pull/8176
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8176: [HUDI-5929] Automatically infer key generator type
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8176:
URL: https://github.com/apache/hudi/pull/8176#issuecomment-1491240123
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3606cf593712177861b1b8eea4ac9dc31ef7dc1c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15708",
"triggerID" : "3606cf593712177861b1b8eea4ac9dc31ef7dc1c",
"triggerType" : "PUSH"
}, {
"hash" : "280559f7b8dc7d1738a7e251b1ad4c42658a48b8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15758",
"triggerID" : "280559f7b8dc7d1738a7e251b1ad4c42658a48b8",
"triggerType" : "PUSH"
}, {
"hash" : "8ac30b3eed9452b6c7ed2715748dd9866770961f",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16002",
"triggerID" : "8ac30b3eed9452b6c7ed2715748dd9866770961f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8ac30b3eed9452b6c7ed2715748dd9866770961f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16002)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8176: [HUDI-5929] Automatically infer key generator type
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8176:
URL: https://github.com/apache/hudi/pull/8176#issuecomment-1472341269
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3606cf593712177861b1b8eea4ac9dc31ef7dc1c",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15708",
"triggerID" : "3606cf593712177861b1b8eea4ac9dc31ef7dc1c",
"triggerType" : "PUSH"
}, {
"hash" : "280559f7b8dc7d1738a7e251b1ad4c42658a48b8",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "280559f7b8dc7d1738a7e251b1ad4c42658a48b8",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3606cf593712177861b1b8eea4ac9dc31ef7dc1c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15708)
* 280559f7b8dc7d1738a7e251b1ad4c42658a48b8 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8176: [HUDI-5929] Automatically infer key generator type
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8176:
URL: https://github.com/apache/hudi/pull/8176#issuecomment-1472412098
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3606cf593712177861b1b8eea4ac9dc31ef7dc1c",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15708",
"triggerID" : "3606cf593712177861b1b8eea4ac9dc31ef7dc1c",
"triggerType" : "PUSH"
}, {
"hash" : "280559f7b8dc7d1738a7e251b1ad4c42658a48b8",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15758",
"triggerID" : "280559f7b8dc7d1738a7e251b1ad4c42658a48b8",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3606cf593712177861b1b8eea4ac9dc31ef7dc1c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15708)
* 280559f7b8dc7d1738a7e251b1ad4c42658a48b8 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15758)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] yihua commented on a diff in pull request #8176: [HUDI-5929] Automatically infer key generator type
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #8176:
URL: https://github.com/apache/hudi/pull/8176#discussion_r1153905604
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/keygen/factory/HoodieSparkKeyGeneratorFactory.java:
##########
@@ -75,40 +79,60 @@ public static KeyGenerator createKeyGenerator(TypedProperties props) throws IOEx
}
}
+ /**
+ * @param type {@link KeyGeneratorType} enum.
+ * @return The key generator class name for Spark based on the {@link KeyGeneratorType}.
+ */
+ public static String getKeyGeneratorClassNameFromType(KeyGeneratorType type) {
+ switch (type) {
+ case SIMPLE:
+ return SimpleKeyGenerator.class.getName();
+ case COMPLEX:
+ return ComplexKeyGenerator.class.getName();
+ case TIMESTAMP:
+ return TimestampBasedKeyGenerator.class.getName();
+ case CUSTOM:
+ return CustomKeyGenerator.class.getName();
+ case NON_PARTITION:
+ return NonpartitionedKeyGenerator.class.getName();
+ case GLOBAL_DELETE:
+ return GlobalDeleteKeyGenerator.class.getName();
+ default:
+ throw new HoodieKeyGeneratorException("Unsupported keyGenerator Type " + type);
+ }
+ }
+
+ /**
+ * Infers the key generator type based on the record key and partition fields.
+ * If neither of the record key and partition fields are set, the default type is returned.
+ *
+ * @param props Properties from the write config.
+ * @return Inferred key generator type.
+ */
+ public static KeyGeneratorType inferKeyGeneratorTypeFromWriteConfig(TypedProperties props) {
Review Comment:
This has to be public, as used by `DataSourceOptionsHelper`.
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/catalyst/catalog/HoodieCatalogTable.scala:
##########
@@ -300,7 +302,7 @@ class HoodieCatalogTable(val spark: SparkSession, var table: CatalogTable) exten
val primaryKeys = table.properties.getOrElse(SQL_KEY_TABLE_PRIMARY_KEY.sqlKeyName, table.storage.properties.get(SQL_KEY_TABLE_PRIMARY_KEY.sqlKeyName)).toString
val partitions = table.partitionColumnNames.mkString(",")
extraConfig(HoodieTableConfig.KEY_GENERATOR_CLASS_NAME.key) =
- DataSourceOptionsHelper.inferKeyGenClazz(primaryKeys, partitions)
+ getKeyGeneratorClassNameFromType(inferKeyGeneratorType(primaryKeys, partitions))
Review Comment:
Fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on a diff in pull request #8176: [HUDI-5929] Automatically infer key generator type
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on code in PR #8176:
URL: https://github.com/apache/hudi/pull/8176#discussion_r1140709444
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/keygen/factory/HoodieSparkKeyGeneratorFactory.java:
##########
@@ -75,40 +79,60 @@ public static KeyGenerator createKeyGenerator(TypedProperties props) throws IOEx
}
}
+ /**
+ * @param type {@link KeyGeneratorType} enum.
+ * @return The key generator class name for Spark based on the {@link KeyGeneratorType}.
+ */
+ public static String getKeyGeneratorClassNameFromType(KeyGeneratorType type) {
+ switch (type) {
+ case SIMPLE:
+ return SimpleKeyGenerator.class.getName();
+ case COMPLEX:
+ return ComplexKeyGenerator.class.getName();
+ case TIMESTAMP:
+ return TimestampBasedKeyGenerator.class.getName();
+ case CUSTOM:
+ return CustomKeyGenerator.class.getName();
+ case NON_PARTITION:
+ return NonpartitionedKeyGenerator.class.getName();
+ case GLOBAL_DELETE:
+ return GlobalDeleteKeyGenerator.class.getName();
+ default:
+ throw new HoodieKeyGeneratorException("Unsupported keyGenerator Type " + type);
+ }
+ }
+
+ /**
+ * Infers the key generator type based on the record key and partition fields.
+ * If neither of the record key and partition fields are set, the default type is returned.
+ *
+ * @param props Properties from the write config.
+ * @return Inferred key generator type.
+ */
+ public static KeyGeneratorType inferKeyGeneratorTypeFromWriteConfig(TypedProperties props) {
Review Comment:
is this required to be public ?
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/catalyst/catalog/HoodieCatalogTable.scala:
##########
@@ -300,7 +302,7 @@ class HoodieCatalogTable(val spark: SparkSession, var table: CatalogTable) exten
val primaryKeys = table.properties.getOrElse(SQL_KEY_TABLE_PRIMARY_KEY.sqlKeyName, table.storage.properties.get(SQL_KEY_TABLE_PRIMARY_KEY.sqlKeyName)).toString
val partitions = table.partitionColumnNames.mkString(",")
extraConfig(HoodieTableConfig.KEY_GENERATOR_CLASS_NAME.key) =
- DataSourceOptionsHelper.inferKeyGenClazz(primaryKeys, partitions)
+ getKeyGeneratorClassNameFromType(inferKeyGeneratorType(primaryKeys, partitions))
Review Comment:
should we introduce InferKeyGenClassfromProps() and internally we can call inferKeyGeneratorType()
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #8176: [HUDI-5929] Automatically infer key generator type
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on PR #8176:
URL: https://github.com/apache/hudi/pull/8176#issuecomment-1474412360
Also, do you think we should fix our quick start to start using key gen type instead of class name.
Also, we might also need to add docs around this auto inference and clarify that users don't need to set key gen class or type for some of the key generators
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8176: [HUDI-5929] Automatically infer key generator type
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8176:
URL: https://github.com/apache/hudi/pull/8176#issuecomment-1491137422
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3606cf593712177861b1b8eea4ac9dc31ef7dc1c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15708",
"triggerID" : "3606cf593712177861b1b8eea4ac9dc31ef7dc1c",
"triggerType" : "PUSH"
}, {
"hash" : "280559f7b8dc7d1738a7e251b1ad4c42658a48b8",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15758",
"triggerID" : "280559f7b8dc7d1738a7e251b1ad4c42658a48b8",
"triggerType" : "PUSH"
}, {
"hash" : "8ac30b3eed9452b6c7ed2715748dd9866770961f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "8ac30b3eed9452b6c7ed2715748dd9866770961f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 280559f7b8dc7d1738a7e251b1ad4c42658a48b8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15758)
* 8ac30b3eed9452b6c7ed2715748dd9866770961f UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8176: [HUDI-5929] Automatically infer key generator type
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8176:
URL: https://github.com/apache/hudi/pull/8176#issuecomment-1472709884
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3606cf593712177861b1b8eea4ac9dc31ef7dc1c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15708",
"triggerID" : "3606cf593712177861b1b8eea4ac9dc31ef7dc1c",
"triggerType" : "PUSH"
}, {
"hash" : "280559f7b8dc7d1738a7e251b1ad4c42658a48b8",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15758",
"triggerID" : "280559f7b8dc7d1738a7e251b1ad4c42658a48b8",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 280559f7b8dc7d1738a7e251b1ad4c42658a48b8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15758)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8176: [HUDI-5929] Automatically infer key generator type
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8176:
URL: https://github.com/apache/hudi/pull/8176#issuecomment-1491140798
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3606cf593712177861b1b8eea4ac9dc31ef7dc1c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15708",
"triggerID" : "3606cf593712177861b1b8eea4ac9dc31ef7dc1c",
"triggerType" : "PUSH"
}, {
"hash" : "280559f7b8dc7d1738a7e251b1ad4c42658a48b8",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15758",
"triggerID" : "280559f7b8dc7d1738a7e251b1ad4c42658a48b8",
"triggerType" : "PUSH"
}, {
"hash" : "8ac30b3eed9452b6c7ed2715748dd9866770961f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16002",
"triggerID" : "8ac30b3eed9452b6c7ed2715748dd9866770961f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 280559f7b8dc7d1738a7e251b1ad4c42658a48b8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15758)
* 8ac30b3eed9452b6c7ed2715748dd9866770961f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16002)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8176: [HUDI-5929] Automatically infer key generator type
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8176:
URL: https://github.com/apache/hudi/pull/8176#issuecomment-1492244623
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8ac30b3eed9452b6c7ed2715748dd9866770961f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "8ac30b3eed9452b6c7ed2715748dd9866770961f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8ac30b3eed9452b6c7ed2715748dd9866770961f UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] yihua commented on pull request #8176: [HUDI-5929] Automatically infer key generator type
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on PR #8176:
URL: https://github.com/apache/hudi/pull/8176#issuecomment-1491133351
> Also, do you think we should fix our quick start to start using key gen type instead of class name. Also, we might also need to add docs around this auto inference and clarify that users don't need to set key gen class or type for some of the key generators
Yes, definitely we should fix docs. I created HUDI-6008 for the docs update.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8176: [HUDI-5929] Automatically infer key generator type
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8176:
URL: https://github.com/apache/hudi/pull/8176#issuecomment-1492496916
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8ac30b3eed9452b6c7ed2715748dd9866770961f",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16002",
"triggerID" : "8ac30b3eed9452b6c7ed2715748dd9866770961f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8ac30b3eed9452b6c7ed2715748dd9866770961f Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16002)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8176: [HUDI-5929] Automatically infer key generator type
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8176:
URL: https://github.com/apache/hudi/pull/8176#issuecomment-1467552859
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3606cf593712177861b1b8eea4ac9dc31ef7dc1c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3606cf593712177861b1b8eea4ac9dc31ef7dc1c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3606cf593712177861b1b8eea4ac9dc31ef7dc1c UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8176: [HUDI-5929] Automatically infer key generator type
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8176:
URL: https://github.com/apache/hudi/pull/8176#issuecomment-1467621160
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3606cf593712177861b1b8eea4ac9dc31ef7dc1c",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15708",
"triggerID" : "3606cf593712177861b1b8eea4ac9dc31ef7dc1c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3606cf593712177861b1b8eea4ac9dc31ef7dc1c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15708)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8176: [HUDI-5929] Automatically infer key generator type
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8176:
URL: https://github.com/apache/hudi/pull/8176#issuecomment-1467815999
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3606cf593712177861b1b8eea4ac9dc31ef7dc1c",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15708",
"triggerID" : "3606cf593712177861b1b8eea4ac9dc31ef7dc1c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3606cf593712177861b1b8eea4ac9dc31ef7dc1c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15708)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org