You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/02/27 09:54:56 UTC

[GitHub] [spark] lyssg opened a new pull request #35667: [K8S] Avoid possible errors due to incorrect file size or type supplied in hadoop conf

lyssg opened a new pull request #35667:
URL: https://github.com/apache/spark/pull/35667


   ### What changes were proposed in this pull request?
   Skip mount files in hadoop conf if they are binary or very large to fit the configMap's max size.
   
   ### Why are the changes needed?
   Config map cannot hold binary files and there is also a limit on how much data a configMap can hold.
   And spark conf limit has been done in SPARK-32221].
   
   ### Does this PR introduce _any_ user-facing change?
   yes, in simple words avoids possible errors due to negligence (for example, placing a large file or a binary file in SPARK_CONF_DIR) and thus improves user experience.
   
   
   ### How was this patch tested?
   Actually testing in k8s cluster and existing tests.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] lyssg commented on pull request #35667: [K8S] Avoid possible errors due to incorrect file size or type supplied in hadoop conf

Posted by GitBox <gi...@apache.org>.
lyssg commented on pull request #35667:
URL: https://github.com/apache/spark/pull/35667#issuecomment-1059960261


   > New test(s) could be added to org.apache.spark.deploy.k8s.features.HadoopConfDriverFeatureStepSuite
   
   ok


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] lyssg commented on pull request #35667: [K8S] Avoid possible errors due to incorrect file size or type supplied in hadoop conf

Posted by GitBox <gi...@apache.org>.
lyssg commented on pull request #35667:
URL: https://github.com/apache/spark/pull/35667#issuecomment-1053761559


   > @lyssg mind linking the JIRA into the PR title please? See also https://spark.apache.org/contributing.html
   
   thanks， i will complete it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #35667: [K8S] Avoid possible errors due to incorrect file size or type supplied in hadoop conf

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #35667:
URL: https://github.com/apache/spark/pull/35667#issuecomment-1053506015


   Can one of the admins verify this patch?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] lyssg commented on pull request #35667: [K8S] Avoid possible errors due to incorrect file size or type supplied in hadoop conf

Posted by GitBox <gi...@apache.org>.
lyssg commented on pull request #35667:
URL: https://github.com/apache/spark/pull/35667#issuecomment-1053525952


   yes
   
   
   
   
   ------------------&nbsp;原始邮件&nbsp;------------------
   发件人: "UCB ***@***.***&gt;; 
   发送时间: 2022年2月27日(星期天) 晚上7:09
   收件人: ***@***.***&gt;; 
   抄送: ***@***.***&gt;; ***@***.***&gt;; 
   主题: Re: [apache/spark] [K8S] Avoid possible errors due to incorrect file size or type supplied in hadoop conf (PR #35667)
   
   
   
   
   
    
   Can one of the admins verify this patch?
    
   —
   Reply to this email directly, view it on GitHub, or unsubscribe.
   Triage notifications on the go with GitHub Mobile for iOS or Android. 
   You are receiving this because you authored the thread.Message ID: ***@***.***&gt;


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on pull request #35667: [K8S] Avoid possible errors due to incorrect file size or type supplied in hadoop conf

Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #35667:
URL: https://github.com/apache/spark/pull/35667#issuecomment-1053752532


   @lyssg mind linking the JIRA into the PR title please? See also https://spark.apache.org/contributing.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org