You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2023/01/17 08:32:22 UTC
[GitHub] [hudi] weimingdiit opened a new pull request, #7684: [HUDI-5567] Modified to make bootstrapping exception message clearer
weimingdiit opened a new pull request, #7684:
URL: https://github.com/apache/hudi/pull/7684
### Change Logs
Exception message maybe can clearer when determine schema from the data files in bootstrap.
### Impact
nothing
### Risk level (write none, low medium or high below)
_If medium or high, explain what verification was done to mitigate the risks._
### Documentation Update
_Describe any necessary documentation update if there is any new feature, config, or user-facing change_
- _The config description must be updated if new configs are added or the default value of the configs are changed_
- _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the
ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make
changes to the website._
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7684: [HUDI-5567] Modified to make bootstrapping exception message clearer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7684:
URL: https://github.com/apache/hudi/pull/7684#issuecomment-1385629825
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "31fe16b17e99594573abc1ad273ee2d007c56bc9",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14360",
"triggerID" : "31fe16b17e99594573abc1ad273ee2d007c56bc9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 31fe16b17e99594573abc1ad273ee2d007c56bc9 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14360)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #7684: [HUDI-5567] Modified to make bootstrapping exception message clearer
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #7684:
URL: https://github.com/apache/hudi/pull/7684#discussion_r1073100473
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/bootstrap/HoodieSparkBootstrapSchemaProvider.java:
##########
@@ -92,7 +92,7 @@ private static Schema getBootstrapSourceSchemaOrc(HoodieWriteConfig writeConfig,
try {
orcReader = OrcFile.createReader(filePath, OrcFile.readerOptions(context.getHadoopConf().get()));
} catch (IOException e) {
- throw new HoodieException("Could not determine schema from the data files.");
+ throw new HoodieException("Could not determine schema from the data files. Create orc reader error.");
}
Review Comment:
`Could not determine schema from the ORC data files.`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7684: [HUDI-5567] Modified to make bootstrapping exception message clearer
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #7684:
URL: https://github.com/apache/hudi/pull/7684#issuecomment-1408000736
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "31fe16b17e99594573abc1ad273ee2d007c56bc9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14360",
"triggerID" : "31fe16b17e99594573abc1ad273ee2d007c56bc9",
"triggerType" : "PUSH"
}, {
"hash" : "5aca03f24eac492f0181657fddfc79e40fd32208",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14754",
"triggerID" : "5aca03f24eac492f0181657fddfc79e40fd32208",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5aca03f24eac492f0181657fddfc79e40fd32208 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14754)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7684: [HUDI-5567] Modified to make bootstrapping exception message clearer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7684:
URL: https://github.com/apache/hudi/pull/7684#issuecomment-1385026811
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "31fe16b17e99594573abc1ad273ee2d007c56bc9",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "31fe16b17e99594573abc1ad273ee2d007c56bc9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 31fe16b17e99594573abc1ad273ee2d007c56bc9 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] weimingdiit commented on a diff in pull request #7684: [HUDI-5567] Modified to make bootstrapping exception message clearer
Posted by "weimingdiit (via GitHub)" <gi...@apache.org>.
weimingdiit commented on code in PR #7684:
URL: https://github.com/apache/hudi/pull/7684#discussion_r1090112497
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/bootstrap/HoodieSparkBootstrapSchemaProvider.java:
##########
@@ -60,7 +60,7 @@ protected Schema getBootstrapSourceSchema(HoodieEngineContext context, List<Pair
} else if (ORC.getFileExtension().equals(extension)) {
return getBootstrapSourceSchemaOrc(writeConfig, context, filePath);
} else {
- throw new HoodieException("Could not determine schema from the data files.");
+ throw new HoodieException("Could not determine schema from the data files. Only two file types, orc and parquet, are supported.");
}
Review Comment:
@danny0405 thanks，danny. i have update this exception message
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/bootstrap/HoodieSparkBootstrapSchemaProvider.java:
##########
@@ -92,7 +92,7 @@ private static Schema getBootstrapSourceSchemaOrc(HoodieWriteConfig writeConfig,
try {
orcReader = OrcFile.createReader(filePath, OrcFile.readerOptions(context.getHadoopConf().get()));
} catch (IOException e) {
- throw new HoodieException("Could not determine schema from the data files.");
+ throw new HoodieException("Could not determine schema from the data files. Create orc reader error.");
}
Review Comment:
@danny0405 thanks，danny. i have update this exception message
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7684: [HUDI-5567] Modified to make bootstrapping exception message clearer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7684:
URL: https://github.com/apache/hudi/pull/7684#issuecomment-1385115523
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "31fe16b17e99594573abc1ad273ee2d007c56bc9",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14360",
"triggerID" : "31fe16b17e99594573abc1ad273ee2d007c56bc9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 31fe16b17e99594573abc1ad273ee2d007c56bc9 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14360)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 merged pull request #7684: [HUDI-5567] Modified to make bootstrapping exception message clearer
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 merged PR #7684:
URL: https://github.com/apache/hudi/pull/7684
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] weimingdiit commented on pull request #7684: [HUDI-5567] Modified to make bootstrapping exception message clearer
Posted by "weimingdiit (via GitHub)" <gi...@apache.org>.
weimingdiit commented on PR #7684:
URL: https://github.com/apache/hudi/pull/7684#issuecomment-1409955146
@danny0405 thanks for your review danny.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on pull request #7684: [HUDI-5567] Modified to make bootstrapping exception message clearer
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on PR #7684:
URL: https://github.com/apache/hudi/pull/7684#issuecomment-1409796950
The failed test is kind of unrelated: https://dev.azure.com/apache-hudi-ci-org/apache-hudi-ci/_build/results?buildId=14754&view=logs&j=dcedfe73-9485-5cc5-817a-73b61fc5dcb0&t=746585d8-b50a-55c3-26c5-517d93af9934&l=40524
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7684: [HUDI-5567] Modified to make bootstrapping exception message clearer
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #7684:
URL: https://github.com/apache/hudi/pull/7684#issuecomment-1407883680
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "31fe16b17e99594573abc1ad273ee2d007c56bc9",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14360",
"triggerID" : "31fe16b17e99594573abc1ad273ee2d007c56bc9",
"triggerType" : "PUSH"
}, {
"hash" : "5aca03f24eac492f0181657fddfc79e40fd32208",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5aca03f24eac492f0181657fddfc79e40fd32208",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 31fe16b17e99594573abc1ad273ee2d007c56bc9 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14360)
* 5aca03f24eac492f0181657fddfc79e40fd32208 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7684: [HUDI-5567] Modified to make bootstrapping exception message clearer
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #7684:
URL: https://github.com/apache/hudi/pull/7684#issuecomment-1407889195
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "31fe16b17e99594573abc1ad273ee2d007c56bc9",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14360",
"triggerID" : "31fe16b17e99594573abc1ad273ee2d007c56bc9",
"triggerType" : "PUSH"
}, {
"hash" : "5aca03f24eac492f0181657fddfc79e40fd32208",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14754",
"triggerID" : "5aca03f24eac492f0181657fddfc79e40fd32208",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 31fe16b17e99594573abc1ad273ee2d007c56bc9 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14360)
* 5aca03f24eac492f0181657fddfc79e40fd32208 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14754)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #7684: [HUDI-5567] Modified to make bootstrapping exception message clearer
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #7684:
URL: https://github.com/apache/hudi/pull/7684#discussion_r1073100251
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/bootstrap/HoodieSparkBootstrapSchemaProvider.java:
##########
@@ -60,7 +60,7 @@ protected Schema getBootstrapSourceSchema(HoodieEngineContext context, List<Pair
} else if (ORC.getFileExtension().equals(extension)) {
return getBootstrapSourceSchemaOrc(writeConfig, context, filePath);
} else {
- throw new HoodieException("Could not determine schema from the data files.");
+ throw new HoodieException("Could not determine schema from the data files. Only two file types, orc and parquet, are supported.");
}
Review Comment:
`Could not determine schema from the data files, supported file formats: [ORC, PARQUET].`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org