You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/02/04 16:07:30 UTC
[GitHub] [spark] ocworld opened a new pull request #35397: [SPARK-38102][CORE] Support custom commitProtocolClass in saveAsNewAPIHadoopDataset
ocworld opened a new pull request #35397:
URL: https://github.com/apache/spark/pull/35397
There is no way to apply spark-hadoop-cloud's commitProtocolClass when using saveAsNewAPIHadoopDataset that are to avoid object storage's problem.
https://spark.apache.org/docs/latest/cloud-integration.html
It is needed to support custom commitProtocolClass class when using saveAsNewAPIHadoopDataset by an option. For example,
```
spark.hadoop.mapreduce.sources.commitProtocolClass org.apache.spark.internal.io.cloud.PathOutputCommitProtocol
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] ocworld commented on pull request #35397: [SPARK-38102][CORE] Support custom commitProtocolClass in saveAsNewAPIHadoopDataset
Posted by GitBox <gi...@apache.org>.
ocworld commented on pull request #35397:
URL: https://github.com/apache/spark/pull/35397#issuecomment-1030128583
@ahnlab-oss
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #35397: [SPARK-38102][CORE] Support custom commitProtocolClass in saveAsNewAPIHadoopDataset
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #35397:
URL: https://github.com/apache/spark/pull/35397#issuecomment-1030338496
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] ocworld commented on pull request #35397: [SPARK-38102][CORE] Support custom commitProtocolClass in saveAsNewAPIHadoopDataset
Posted by GitBox <gi...@apache.org>.
ocworld commented on pull request #35397:
URL: https://github.com/apache/spark/pull/35397#issuecomment-1030581934
After fixing unittest error, I will recreate pr.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] ocworld closed pull request #35397: [SPARK-38102][CORE] Support custom commitProtocolClass in saveAsNewAPIHadoopDataset
Posted by GitBox <gi...@apache.org>.
ocworld closed pull request #35397:
URL: https://github.com/apache/spark/pull/35397
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org