You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/11/04 18:28:28 UTC
[GitHub] [hudi] nsivabalan opened a new pull request, #7140: [HUDI-5163] Fixing failure handling with spark datasource write
nsivabalan opened a new pull request, #7140:
URL: https://github.com/apache/hudi/pull/7140
### Change Logs
Looks like in DefaultSource, we don't fail the write if there are any failure. for most failures, we throw all the way. but if write status had failures or other reasons, we don't actually fail the spark command.
### Impact
spark write failures will fail the command for all valid failure scenarios.
### Risk level (write none, low medium or high below)
low.
### Documentation Update
N/A
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7140: [HUDI-5163] Fixing failure handling with spark datasource write
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7140:
URL: https://github.com/apache/hudi/pull/7140#issuecomment-1304041805
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c91d6757e0788b8b06f8ac011776a248d6b6b449",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12794",
"triggerID" : "c91d6757e0788b8b06f8ac011776a248d6b6b449",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c91d6757e0788b8b06f8ac011776a248d6b6b449 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12794)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7140: [HUDI-5163] Fixing failure handling with spark datasource write
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7140:
URL: https://github.com/apache/hudi/pull/7140#issuecomment-1338557880
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c91d6757e0788b8b06f8ac011776a248d6b6b449",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12794",
"triggerID" : "c91d6757e0788b8b06f8ac011776a248d6b6b449",
"triggerType" : "PUSH"
}, {
"hash" : "ba949eea9cf6222c2bcd0355bbb79b7765bf381c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "ba949eea9cf6222c2bcd0355bbb79b7765bf381c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c91d6757e0788b8b06f8ac011776a248d6b6b449 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12794)
* ba949eea9cf6222c2bcd0355bbb79b7765bf381c UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #7140: [HUDI-5163] Fixing failure handling with spark datasource write
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on PR #7140:
URL: https://github.com/apache/hudi/pull/7140#issuecomment-1303984715
@YannByron : I might need some help to take this patch and make valid fixes for spark-sql classes.
Also, we might need to write tests. if you can loop in someone, would be nice.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7140: [HUDI-5163] Fixing failure handling with spark datasource write
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7140:
URL: https://github.com/apache/hudi/pull/7140#issuecomment-1340096796
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c91d6757e0788b8b06f8ac011776a248d6b6b449",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12794",
"triggerID" : "c91d6757e0788b8b06f8ac011776a248d6b6b449",
"triggerType" : "PUSH"
}, {
"hash" : "ba949eea9cf6222c2bcd0355bbb79b7765bf381c",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13469",
"triggerID" : "ba949eea9cf6222c2bcd0355bbb79b7765bf381c",
"triggerType" : "PUSH"
}, {
"hash" : "d765a6c72b4195ab1131af3210b942d5e2cb3880",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13488",
"triggerID" : "d765a6c72b4195ab1131af3210b942d5e2cb3880",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ba949eea9cf6222c2bcd0355bbb79b7765bf381c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13469)
* d765a6c72b4195ab1131af3210b942d5e2cb3880 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13488)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] Zouxxyy commented on pull request #7140: [HUDI-5163] Fixing failure handling with spark datasource write
Posted by GitBox <gi...@apache.org>.
Zouxxyy commented on PR #7140:
URL: https://github.com/apache/hudi/pull/7140#issuecomment-1319635657
@nsivabalan I made a [fix](https://github.com/nsivabalan/hudi/pull/12) based on the comments, hope that helps, but sorry couldn't find a concrete test case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7140: [HUDI-5163] Fixing failure handling with spark datasource write
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7140:
URL: https://github.com/apache/hudi/pull/7140#issuecomment-1340231372
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c91d6757e0788b8b06f8ac011776a248d6b6b449",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12794",
"triggerID" : "c91d6757e0788b8b06f8ac011776a248d6b6b449",
"triggerType" : "PUSH"
}, {
"hash" : "ba949eea9cf6222c2bcd0355bbb79b7765bf381c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13469",
"triggerID" : "ba949eea9cf6222c2bcd0355bbb79b7765bf381c",
"triggerType" : "PUSH"
}, {
"hash" : "d765a6c72b4195ab1131af3210b942d5e2cb3880",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13488",
"triggerID" : "d765a6c72b4195ab1131af3210b942d5e2cb3880",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* d765a6c72b4195ab1131af3210b942d5e2cb3880 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13488)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7140: [HUDI-5163] Fixing failure handling with spark datasource write
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7140:
URL: https://github.com/apache/hudi/pull/7140#issuecomment-1340086541
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c91d6757e0788b8b06f8ac011776a248d6b6b449",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12794",
"triggerID" : "c91d6757e0788b8b06f8ac011776a248d6b6b449",
"triggerType" : "PUSH"
}, {
"hash" : "ba949eea9cf6222c2bcd0355bbb79b7765bf381c",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13469",
"triggerID" : "ba949eea9cf6222c2bcd0355bbb79b7765bf381c",
"triggerType" : "PUSH"
}, {
"hash" : "d765a6c72b4195ab1131af3210b942d5e2cb3880",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d765a6c72b4195ab1131af3210b942d5e2cb3880",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ba949eea9cf6222c2bcd0355bbb79b7765bf381c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13469)
* d765a6c72b4195ab1131af3210b942d5e2cb3880 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] YannByron commented on pull request #7140: [HUDI-5163] Fixing failure handling with spark datasource write
Posted by GitBox <gi...@apache.org>.
YannByron commented on PR #7140:
URL: https://github.com/apache/hudi/pull/7140#issuecomment-1308132400
> @YannByron : I might need some help to take this patch and make valid fixes for spark-sql classes. Also, we might need to write tests. if you can loop in someone, would be nice.
@Zouxxyy would you like to take this up ? If yes, you can open another pr based on this pr and comments for your convenience.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] Zouxxyy commented on pull request #7140: [HUDI-5163] Fixing failure handling with spark datasource write
Posted by GitBox <gi...@apache.org>.
Zouxxyy commented on PR #7140:
URL: https://github.com/apache/hudi/pull/7140#issuecomment-1309666320
> > @YannByron : I might need some help to take this patch and make valid fixes for spark-sql classes. Also, we might need to write tests. if you can loop in someone, would be nice.
>
> @Zouxxyy would you like to take this up to enrich more cases to validate? If yes, you can open another pr based on this pr and comments for your convenience.
@YannByron Ok, I'll fix it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope merged pull request #7140: [HUDI-5163] Fixing failure handling with spark datasource write
Posted by GitBox <gi...@apache.org>.
codope merged PR #7140:
URL: https://github.com/apache/hudi/pull/7140
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7140: [HUDI-5163] Fixing failure handling with spark datasource write
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7140:
URL: https://github.com/apache/hudi/pull/7140#issuecomment-1304282046
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c91d6757e0788b8b06f8ac011776a248d6b6b449",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12794",
"triggerID" : "c91d6757e0788b8b06f8ac011776a248d6b6b449",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c91d6757e0788b8b06f8ac011776a248d6b6b449 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12794)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7140: [HUDI-5163] Fixing failure handling with spark datasource write
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7140:
URL: https://github.com/apache/hudi/pull/7140#issuecomment-1304026365
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c91d6757e0788b8b06f8ac011776a248d6b6b449",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "c91d6757e0788b8b06f8ac011776a248d6b6b449",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c91d6757e0788b8b06f8ac011776a248d6b6b449 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #7140: [HUDI-5163] Fixing failure handling with spark datasource write
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on PR #7140:
URL: https://github.com/apache/hudi/pull/7140#issuecomment-1338205028
Addressed all comments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7140: [HUDI-5163] Fixing failure handling with spark datasource write
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7140:
URL: https://github.com/apache/hudi/pull/7140#issuecomment-1338592223
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c91d6757e0788b8b06f8ac011776a248d6b6b449",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12794",
"triggerID" : "c91d6757e0788b8b06f8ac011776a248d6b6b449",
"triggerType" : "PUSH"
}, {
"hash" : "ba949eea9cf6222c2bcd0355bbb79b7765bf381c",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13469",
"triggerID" : "ba949eea9cf6222c2bcd0355bbb79b7765bf381c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c91d6757e0788b8b06f8ac011776a248d6b6b449 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12794)
* ba949eea9cf6222c2bcd0355bbb79b7765bf381c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13469)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7140: [HUDI-5163] Fixing failure handling with spark datasource write
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7140:
URL: https://github.com/apache/hudi/pull/7140#issuecomment-1338609080
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c91d6757e0788b8b06f8ac011776a248d6b6b449",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12794",
"triggerID" : "c91d6757e0788b8b06f8ac011776a248d6b6b449",
"triggerType" : "PUSH"
}, {
"hash" : "ba949eea9cf6222c2bcd0355bbb79b7765bf381c",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13469",
"triggerID" : "ba949eea9cf6222c2bcd0355bbb79b7765bf381c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ba949eea9cf6222c2bcd0355bbb79b7765bf381c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13469)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #7140: [HUDI-5163] Fixing failure handling with spark datasource write
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on code in PR #7140:
URL: https://github.com/apache/hudi/pull/7140#discussion_r1015901847
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DefaultSource.scala:
##########
@@ -144,11 +144,15 @@ class DefaultSource extends RelationProvider
if (optParams.get(OPERATION.key).contains(BOOTSTRAP_OPERATION_OPT_VAL)) {
HoodieSparkSqlWriter.bootstrap(sqlContext, mode, optParams, dfWithoutMetaCols)
+ HoodieSparkSqlWriter.cleanup()
} else {
- HoodieSparkSqlWriter.write(sqlContext, mode, optParams, dfWithoutMetaCols)
+ val succeeded = HoodieSparkSqlWriter.write(sqlContext, mode, optParams, dfWithoutMetaCols)._1
Review Comment:
Let's un-tuple instead:
```
val (_, succeeded) = ...
```
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/command/AlterHoodieTableDropPartitionCommand.scala:
##########
@@ -59,11 +60,13 @@ case class AlterHoodieTableDropPartitionCommand(
// delete partition files by enabling cleaner and setting retention policies.
val partitionsToDrop = getPartitionPathToDrop(hoodieCatalogTable, normalizedSpecs)
val parameters = buildHoodieDropPartitionsConfig(sparkSession, hoodieCatalogTable, partitionsToDrop)
- HoodieSparkSqlWriter.write(
+ if (!HoodieSparkSqlWriter.write(
Review Comment:
Same as above
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DefaultSource.scala:
##########
@@ -144,11 +144,15 @@ class DefaultSource extends RelationProvider
if (optParams.get(OPERATION.key).contains(BOOTSTRAP_OPERATION_OPT_VAL)) {
HoodieSparkSqlWriter.bootstrap(sqlContext, mode, optParams, dfWithoutMetaCols)
+ HoodieSparkSqlWriter.cleanup()
} else {
- HoodieSparkSqlWriter.write(sqlContext, mode, optParams, dfWithoutMetaCols)
+ val succeeded = HoodieSparkSqlWriter.write(sqlContext, mode, optParams, dfWithoutMetaCols)._1
Review Comment:
Please ditto for all
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] KnightChess commented on pull request #7140: [HUDI-5163] Fixing failure handling with spark datasource write
Posted by GitBox <gi...@apache.org>.
KnightChess commented on PR #7140:
URL: https://github.com/apache/hudi/pull/7140#issuecomment-1316387791
@Zouxxyy hi, this is not a problem of stable recovery and I didn't save the case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org