You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/09/08 08:49:46 UTC
[GitHub] [hudi] TJX2014 opened a new pull request, #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
TJX2014 opened a new pull request, #6634:
URL: https://github.com/apache/hudi/pull/6634
### Change Logs
Fix KEY_GENERATOR_CLASS_NAME will be assigned default as `ComplexKeyGenerator` in `org.apache.spark.sql.catalyst.catalog.HoodieCatalogTable#extraTableConfig`
The `org.apache.hudi.DataSourceOptionsHelper#inferKeyGenClazz` is not used at all：
at `org.apache.hudi.DataSourceWriteOptions#keyGeneraterInferFunc`, because not set default value, will not be called；at `org.apache.spark.sql.hudi.command.SqlKeyGenerator#getRealKeyGenClassName`, also not be called, because `beforeKeyGenClassName != null && beforeKeyGenClassName.nonEmpty` is excluded from `org.apache.spark.sql.hudi.command.SqlKeyGenerator#originKeyGen`
### Impact
SparkSql will also follow infer way when KEY_GENERATOR_CLASS_NAME not user defined.
**Risk level: none | low | medium | high**
none
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] TJX2014 commented on a diff in pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
TJX2014 commented on code in PR #6634:
URL: https://github.com/apache/hudi/pull/6634#discussion_r966746425
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala:
##########
@@ -788,11 +788,11 @@ object DataSourceOptionsHelper {
def inferKeyGenClazz(props: TypedProperties): String = {
val partitionFields = props.getString(DataSourceWriteOptions.PARTITIONPATH_FIELD.key(), null)
val recordsKeyFields = props.getString(DataSourceWriteOptions.RECORDKEY_FIELD.key(), DataSourceWriteOptions.RECORDKEY_FIELD.defaultValue())
- genKeyGenerator(recordsKeyFields, partitionFields)
+ inferKeyGenClazz(recordsKeyFields, partitionFields)
}
- def genKeyGenerator(recordsKeyFields: String, partitionFields: String): String = {
- if (partitionFields != null) {
+ def inferKeyGenClazz(recordsKeyFields: String, partitionFields: String): String = {
+ if (!StringUtils.isNullOrEmpty(partitionFields)) {
Review Comment:
Hi @danny0405 , seems we should judge empty "" for this, else we cannot get `NonpartitionedKeyGenerator` in any situation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] TJX2014 commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
TJX2014 commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1247975433
Hi, @danny0405，https://dev.azure.com/apache-hudi-ci-org/apache-hudi-ci/_build/results?buildId=11379&view=results this is the latest right, seems ci will retain at least one failed record：
https://github.com/apache/hudi/commit/81546636356aa5b224b87087bad14a1da1a06a4d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11315)
https://github.com/apache/hudi/commit/f32cf8515a4984c75bc26641abd3a5b042ebe372 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11379)
FYI : )
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1244998507
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11251",
"triggerID" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"triggerType" : "PUSH"
}, {
"hash" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11266",
"triggerID" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"triggerType" : "PUSH"
}, {
"hash" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11266",
"triggerID" : "1244848914",
"triggerType" : "MANUAL"
}, {
"hash" : "81546636356aa5b224b87087bad14a1da1a06a4d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11315",
"triggerID" : "81546636356aa5b224b87087bad14a1da1a06a4d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 81546636356aa5b224b87087bad14a1da1a06a4d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11315)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
danny0405 commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1244848914
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] TJX2014 commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
TJX2014 commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1240518290
I will check this ci failed issue local.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
danny0405 commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1247696507
The CI shows that `TestSavepointsProcedure#Test Call rollback_to_savepoint Procedure` is still failing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
danny0405 commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1244962825
Still some test failure here, can you check again ~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1244860162
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11251",
"triggerID" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"triggerType" : "PUSH"
}, {
"hash" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11266",
"triggerID" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"triggerType" : "PUSH"
}, {
"hash" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11266",
"triggerID" : "1244848914",
"triggerType" : "MANUAL"
}, {
"hash" : "81546636356aa5b224b87087bad14a1da1a06a4d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "81546636356aa5b224b87087bad14a1da1a06a4d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 019fa62b310060a03b31afd18c159543b24ac4c0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11266)
* 81546636356aa5b224b87087bad14a1da1a06a4d UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1247914631
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11251",
"triggerID" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"triggerType" : "PUSH"
}, {
"hash" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11266",
"triggerID" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"triggerType" : "PUSH"
}, {
"hash" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11266",
"triggerID" : "1244848914",
"triggerType" : "MANUAL"
}, {
"hash" : "81546636356aa5b224b87087bad14a1da1a06a4d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11315",
"triggerID" : "81546636356aa5b224b87087bad14a1da1a06a4d",
"triggerType" : "PUSH"
}, {
"hash" : "f9bceeec2817bcaf5680e58b44b3d948060c36cd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11350",
"triggerID" : "f9bceeec2817bcaf5680e58b44b3d948060c36cd",
"triggerType" : "PUSH"
}, {
"hash" : "81546636356aa5b224b87087bad14a1da1a06a4d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11315",
"triggerID" : "1246501724",
"triggerType" : "MANUAL"
}, {
"hash" : "f32cf8515a4984c75bc26641abd3a5b042ebe372",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "f32cf8515a4984c75bc26641abd3a5b042ebe372",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 81546636356aa5b224b87087bad14a1da1a06a4d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11315)
* f32cf8515a4984c75bc26641abd3a5b042ebe372 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1246809109
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11251",
"triggerID" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"triggerType" : "PUSH"
}, {
"hash" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11266",
"triggerID" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"triggerType" : "PUSH"
}, {
"hash" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11266",
"triggerID" : "1244848914",
"triggerType" : "MANUAL"
}, {
"hash" : "81546636356aa5b224b87087bad14a1da1a06a4d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11315",
"triggerID" : "81546636356aa5b224b87087bad14a1da1a06a4d",
"triggerType" : "PUSH"
}, {
"hash" : "f9bceeec2817bcaf5680e58b44b3d948060c36cd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11350",
"triggerID" : "f9bceeec2817bcaf5680e58b44b3d948060c36cd",
"triggerType" : "PUSH"
}, {
"hash" : "81546636356aa5b224b87087bad14a1da1a06a4d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11315",
"triggerID" : "1246501724",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 81546636356aa5b224b87087bad14a1da1a06a4d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11315)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] TJX2014 commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
TJX2014 commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1240430808
Hi @danny0405 , after my test, I find the inferFunc of https://github.com/apache/hudi/pull/5815 not be called, please correct me if I am not right.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #6634:
URL: https://github.com/apache/hudi/pull/6634#discussion_r966556852
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala:
##########
@@ -787,9 +787,13 @@ object DataSourceOptionsHelper {
def inferKeyGenClazz(props: TypedProperties): String = {
val partitionFields = props.getString(DataSourceWriteOptions.PARTITIONPATH_FIELD.key(), null)
+ val recordsKeyFields = props.getString(DataSourceWriteOptions.RECORDKEY_FIELD.key(), DataSourceWriteOptions.RECORDKEY_FIELD.defaultValue())
+ genKeyGenerator(recordsKeyFields, partitionFields)
+ }
+
+ def genKeyGenerator(recordsKeyFields: String, partitionFields: String): String = {
if (partitionFields != null) {
Review Comment:
Let just rename `genKeyGenerator` to `inferKeyGenClazz`, it's okey because they have different method signature. Also can we write a test case for Spark SQL then ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
danny0405 commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1244814343
cc @TJX2014 , nice ping, you may need to check the CI failure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
danny0405 commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1246301767
> A little hard to reproduce, please give me some time.
No hurry, thanks for the patience.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
danny0405 commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1242590550
Hi @TJX2014 , the PR overall looks good, but there are test failures, can you check and take a look ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 merged pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
danny0405 merged PR #6634:
URL: https://github.com/apache/hudi/pull/6634
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1244857521
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11251",
"triggerID" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"triggerType" : "PUSH"
}, {
"hash" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11266",
"triggerID" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"triggerType" : "PUSH"
}, {
"hash" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1244848914",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 019fa62b310060a03b31afd18c159543b24ac4c0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11266)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1247920497
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11251",
"triggerID" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"triggerType" : "PUSH"
}, {
"hash" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11266",
"triggerID" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"triggerType" : "PUSH"
}, {
"hash" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11266",
"triggerID" : "1244848914",
"triggerType" : "MANUAL"
}, {
"hash" : "81546636356aa5b224b87087bad14a1da1a06a4d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11315",
"triggerID" : "81546636356aa5b224b87087bad14a1da1a06a4d",
"triggerType" : "PUSH"
}, {
"hash" : "f9bceeec2817bcaf5680e58b44b3d948060c36cd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11350",
"triggerID" : "f9bceeec2817bcaf5680e58b44b3d948060c36cd",
"triggerType" : "PUSH"
}, {
"hash" : "81546636356aa5b224b87087bad14a1da1a06a4d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11315",
"triggerID" : "1246501724",
"triggerType" : "MANUAL"
}, {
"hash" : "f32cf8515a4984c75bc26641abd3a5b042ebe372",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11379",
"triggerID" : "f32cf8515a4984c75bc26641abd3a5b042ebe372",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 81546636356aa5b224b87087bad14a1da1a06a4d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11315)
* f32cf8515a4984c75bc26641abd3a5b042ebe372 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11379)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1244862816
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11251",
"triggerID" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"triggerType" : "PUSH"
}, {
"hash" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11266",
"triggerID" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"triggerType" : "PUSH"
}, {
"hash" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11266",
"triggerID" : "1244848914",
"triggerType" : "MANUAL"
}, {
"hash" : "81546636356aa5b224b87087bad14a1da1a06a4d",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11315",
"triggerID" : "81546636356aa5b224b87087bad14a1da1a06a4d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 019fa62b310060a03b31afd18c159543b24ac4c0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11266)
* 81546636356aa5b224b87087bad14a1da1a06a4d Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11315)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #6634:
URL: https://github.com/apache/hudi/pull/6634#discussion_r967562493
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala:
##########
@@ -788,11 +788,11 @@ object DataSourceOptionsHelper {
def inferKeyGenClazz(props: TypedProperties): String = {
val partitionFields = props.getString(DataSourceWriteOptions.PARTITIONPATH_FIELD.key(), null)
val recordsKeyFields = props.getString(DataSourceWriteOptions.RECORDKEY_FIELD.key(), DataSourceWriteOptions.RECORDKEY_FIELD.defaultValue())
- genKeyGenerator(recordsKeyFields, partitionFields)
+ inferKeyGenClazz(recordsKeyFields, partitionFields)
}
- def genKeyGenerator(recordsKeyFields: String, partitionFields: String): String = {
- if (partitionFields != null) {
+ def inferKeyGenClazz(recordsKeyFields: String, partitionFields: String): String = {
+ if (!StringUtils.isNullOrEmpty(partitionFields)) {
Review Comment:
Sure, we need to do that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] TJX2014 commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
TJX2014 commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1246501724
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1240477132
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1a7511e07003745ea2d0d7a802ea5bf1731bd9a4 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] TJX2014 commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
TJX2014 commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1247515534
@danny0405 Please help me review, seems ok in the last ci test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] TJX2014 commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
TJX2014 commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1247514620
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1241753071
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11251",
"triggerID" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"triggerType" : "PUSH"
}, {
"hash" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11266",
"triggerID" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 019fa62b310060a03b31afd18c159543b24ac4c0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11266)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1246535823
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11251",
"triggerID" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"triggerType" : "PUSH"
}, {
"hash" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11266",
"triggerID" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"triggerType" : "PUSH"
}, {
"hash" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11266",
"triggerID" : "1244848914",
"triggerType" : "MANUAL"
}, {
"hash" : "81546636356aa5b224b87087bad14a1da1a06a4d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11315",
"triggerID" : "81546636356aa5b224b87087bad14a1da1a06a4d",
"triggerType" : "PUSH"
}, {
"hash" : "f9bceeec2817bcaf5680e58b44b3d948060c36cd",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11350",
"triggerID" : "f9bceeec2817bcaf5680e58b44b3d948060c36cd",
"triggerType" : "PUSH"
}, {
"hash" : "81546636356aa5b224b87087bad14a1da1a06a4d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1246501724",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 81546636356aa5b224b87087bad14a1da1a06a4d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11315)
* f9bceeec2817bcaf5680e58b44b3d948060c36cd Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11350)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] TJX2014 commented on a diff in pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
TJX2014 commented on code in PR #6634:
URL: https://github.com/apache/hudi/pull/6634#discussion_r966590414
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala:
##########
@@ -787,9 +787,13 @@ object DataSourceOptionsHelper {
def inferKeyGenClazz(props: TypedProperties): String = {
val partitionFields = props.getString(DataSourceWriteOptions.PARTITIONPATH_FIELD.key(), null)
+ val recordsKeyFields = props.getString(DataSourceWriteOptions.RECORDKEY_FIELD.key(), DataSourceWriteOptions.RECORDKEY_FIELD.defaultValue())
+ genKeyGenerator(recordsKeyFields, partitionFields)
+ }
+
+ def genKeyGenerator(recordsKeyFields: String, partitionFields: String): String = {
if (partitionFields != null) {
Review Comment:
Ok, I will do it right now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1241689360
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11251",
"triggerID" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"triggerType" : "PUSH"
}, {
"hash" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11266",
"triggerID" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1a7511e07003745ea2d0d7a802ea5bf1731bd9a4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11251)
* 019fa62b310060a03b31afd18c159543b24ac4c0 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11266)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] TJX2014 commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
TJX2014 commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1240623881
Seeems `TestHoodieSparkSqlWriter.testToWriteWithoutParametersIncludedInHoodieTableConfig:868 » NoSuchElement` not related to this pr.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1246376452
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11251",
"triggerID" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"triggerType" : "PUSH"
}, {
"hash" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11266",
"triggerID" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"triggerType" : "PUSH"
}, {
"hash" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11266",
"triggerID" : "1244848914",
"triggerType" : "MANUAL"
}, {
"hash" : "81546636356aa5b224b87087bad14a1da1a06a4d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11315",
"triggerID" : "81546636356aa5b224b87087bad14a1da1a06a4d",
"triggerType" : "PUSH"
}, {
"hash" : "f9bceeec2817bcaf5680e58b44b3d948060c36cd",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "f9bceeec2817bcaf5680e58b44b3d948060c36cd",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 81546636356aa5b224b87087bad14a1da1a06a4d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11315)
* f9bceeec2817bcaf5680e58b44b3d948060c36cd UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] TJX2014 commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
TJX2014 commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1244823600
@danny0405 Ok, I will check the CI right now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
danny0405 commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1247771867
Can you rebase with the latest master code and force push to re-trigger the tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1246381964
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11251",
"triggerID" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"triggerType" : "PUSH"
}, {
"hash" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11266",
"triggerID" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"triggerType" : "PUSH"
}, {
"hash" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11266",
"triggerID" : "1244848914",
"triggerType" : "MANUAL"
}, {
"hash" : "81546636356aa5b224b87087bad14a1da1a06a4d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11315",
"triggerID" : "81546636356aa5b224b87087bad14a1da1a06a4d",
"triggerType" : "PUSH"
}, {
"hash" : "f9bceeec2817bcaf5680e58b44b3d948060c36cd",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11350",
"triggerID" : "f9bceeec2817bcaf5680e58b44b3d948060c36cd",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 81546636356aa5b224b87087bad14a1da1a06a4d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11315)
* f9bceeec2817bcaf5680e58b44b3d948060c36cd Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11350)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] TJX2014 commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
TJX2014 commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1246152006
A little hard to reproduce, please give me some time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] TJX2014 commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
TJX2014 commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1248881741
Hi @danny0405 ci passed : )
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] TJX2014 commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
TJX2014 commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1244997072
Ok, I will fix it right now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1248071289
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11251",
"triggerID" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"triggerType" : "PUSH"
}, {
"hash" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11266",
"triggerID" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"triggerType" : "PUSH"
}, {
"hash" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11266",
"triggerID" : "1244848914",
"triggerType" : "MANUAL"
}, {
"hash" : "81546636356aa5b224b87087bad14a1da1a06a4d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11315",
"triggerID" : "81546636356aa5b224b87087bad14a1da1a06a4d",
"triggerType" : "PUSH"
}, {
"hash" : "f9bceeec2817bcaf5680e58b44b3d948060c36cd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11350",
"triggerID" : "f9bceeec2817bcaf5680e58b44b3d948060c36cd",
"triggerType" : "PUSH"
}, {
"hash" : "81546636356aa5b224b87087bad14a1da1a06a4d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11315",
"triggerID" : "1246501724",
"triggerType" : "MANUAL"
}, {
"hash" : "f32cf8515a4984c75bc26641abd3a5b042ebe372",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11379",
"triggerID" : "f32cf8515a4984c75bc26641abd3a5b042ebe372",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* f32cf8515a4984c75bc26641abd3a5b042ebe372 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11379)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1246542473
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11251",
"triggerID" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"triggerType" : "PUSH"
}, {
"hash" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11266",
"triggerID" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"triggerType" : "PUSH"
}, {
"hash" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11266",
"triggerID" : "1244848914",
"triggerType" : "MANUAL"
}, {
"hash" : "81546636356aa5b224b87087bad14a1da1a06a4d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11315",
"triggerID" : "81546636356aa5b224b87087bad14a1da1a06a4d",
"triggerType" : "PUSH"
}, {
"hash" : "f9bceeec2817bcaf5680e58b44b3d948060c36cd",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11350",
"triggerID" : "f9bceeec2817bcaf5680e58b44b3d948060c36cd",
"triggerType" : "PUSH"
}, {
"hash" : "81546636356aa5b224b87087bad14a1da1a06a4d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11315",
"triggerID" : "1246501724",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 81546636356aa5b224b87087bad14a1da1a06a4d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11315)
* f9bceeec2817bcaf5680e58b44b3d948060c36cd Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11350)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] TJX2014 commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
TJX2014 commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1247733905
> TestSavepointsProcedure
Hi @danny0405 seems the failed ci record is triggered in two days ago, the failed test is：
- Test Create Hoodie Table With Options *** FAILED ***
Expected "....apache.hudi.keygen.[Complex]KeyGenerator", but got "....apache.hudi.keygen.[Simple]KeyGenerator" (TestCreateTable.scala:139)
Total number of tests run: 186
Suites: completed 46, aborted 0
Tests: succeeded 185, failed 1, canceled 0, ignored 0, pending 0
Seems not the recent one, the failed one is what I have corrected yesterday.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] TJX2014 commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
TJX2014 commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1247832138
> Can you rebase with the latest master code and force push to re-trigger the tests.
Ok.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1241683397
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11251",
"triggerID" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"triggerType" : "PUSH"
}, {
"hash" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "019fa62b310060a03b31afd18c159543b24ac4c0",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1a7511e07003745ea2d0d7a802ea5bf1731bd9a4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11251)
* 019fa62b310060a03b31afd18c159543b24ac4c0 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1240483194
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11251",
"triggerID" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1a7511e07003745ea2d0d7a802ea5bf1731bd9a4 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11251)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6634:
URL: https://github.com/apache/hudi/pull/6634#issuecomment-1241053170
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11251",
"triggerID" : "1a7511e07003745ea2d0d7a802ea5bf1731bd9a4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1a7511e07003745ea2d0d7a802ea5bf1731bd9a4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11251)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org