You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/02/07 07:57:48 UTC
[GitHub] [spark] ocworld opened a new pull request #35417: [SPARK-38102][CORE] Support custom commitProtocolClass in saveAsNewAPIHadoopDataset
ocworld opened a new pull request #35417:
URL: https://github.com/apache/spark/pull/35417
There is no way to apply spark-hadoop-cloud's commitProtocolClass when using saveAsNewAPIHadoopDataset that are to avoid object storage's problem.
https://spark.apache.org/docs/latest/cloud-integration.html
It is needed to support custom commitProtocolClass class when using saveAsNewAPIHadoopDataset by an option. For example,
```
spark.hadoop.mapreduce.sources.commitProtocolClass org.apache.spark.internal.io.cloud.PathOutputCommitProtocol
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #35417: [SPARK-38102][CORE] Support custom commitProtocolClass in saveAsNewAPIHadoopDataset
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #35417:
URL: https://github.com/apache/spark/pull/35417#issuecomment-1031931949
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org