You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/05/14 17:07:03 UTC

[GitHub] [spark] karuppayya opened a new pull request #28529: [SPARK-31692][SQL] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

karuppayya opened a new pull request #28529:
URL: https://github.com/apache/spark/pull/28529


   Pass hadoop confs  specifed via Spark confs to URLStreamHandlerfactory
   
   **BEFORE**
   ```
   ➜  spark git:(SPARK-31692) ✗ ./bin/spark-shell --conf spark.hadoop.fs.file.impl=org.apache.hadoop.fs.RawLocalFileSystem
   
   scala> spark.sharedState
   res0: org.apache.spark.sql.internal.SharedState = org.apache.spark.sql.internal.SharedState5793cd84
   
   scala> new java.net.URL("file:///tmp/1.txt").openConnection.getInputStream
   res1: java.io.InputStream = org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream22846025
   
   scala> import org.apache.hadoop.fs._
   import org.apache.hadoop.fs._
   
   scala>  FileSystem.get(new Path("file:///tmp/1.txt").toUri, spark.sparkContext.hadoopConfiguration)
   res2: org.apache.hadoop.fs.FileSystem = org.apache.hadoop.fs.LocalFileSystem5a930c03
   ```
   
   **AFTER**
   ```
   ➜  spark git:(SPARK-31692) ✗ ./bin/spark-shell --conf spark.hadoop.fs.file.impl=org.apache.hadoop.fs.RawLocalFileSystem
   
   scala> spark.sharedState
   res0: org.apache.spark.sql.internal.SharedState = org.apache.spark.sql.internal.SharedState5c24a636
   
   scala> new java.net.URL("file:///tmp/1.txt").openConnection.getInputStream
   res1: java.io.InputStream = org.apache.hadoop.fs.FSDataInputStream2ba8f528
   
   scala> import org.apache.hadoop.fs._
   import org.apache.hadoop.fs._
   
   scala>  FileSystem.get(new Path("file:///tmp/1.txt").toUri, spark.sparkContext.hadoopConfiguration)
   res2: org.apache.hadoop.fs.FileSystem = LocalFS
   
   scala>  FileSystem.get(new Path("file:///tmp/1.txt").toUri, spark.sparkContext.hadoopConfiguration).getClass
   res3: Class[_ <: org.apache.hadoop.fs.FileSystem] = class org.apache.hadoop.fs.RawLocalFileSystem
   ```
   The type of FileSystem object created(you can check the last statement in the above snippets) in the above two cases are different, which should not have been the case
   
   No
   
   Tested locally.
   Added Unit test
   
   Closes #28516 from karuppayya/SPARK-31692.
   
   Authored-by: Karuppayya Rajendran <ka...@gmail.com>
   Signed-off-by: Dongjoon Hyun <do...@apache.org>
   (cherry picked from commit 72601460ada41761737f39d5dff8e69444fce2ba)
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] karuppayya commented on a change in pull request #28529: [SPARK-31692][SQL] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
karuppayya commented on a change in pull request #28529:
URL: https://github.com/apache/spark/pull/28529#discussion_r425315890



##########
File path: sql/core/src/main/scala/org/apache/spark/sql/internal/SharedState.scala
##########
@@ -157,7 +157,13 @@ private[sql] class SharedState(val sparkContext: SparkContext) extends Logging {
 
 object SharedState extends Logging {
   try {
-    URL.setURLStreamHandlerFactory(new FsUrlStreamHandlerFactory())
+    SparkSession.getActiveSession match {

Review comment:
       Yes, we dont have the configuration patch




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #28529: [SPARK-31692][SQL] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28529:
URL: https://github.com/apache/spark/pull/28529#issuecomment-628790360






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #28529: [SPARK-31692][SQL] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28529:
URL: https://github.com/apache/spark/pull/28529#issuecomment-628790360






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] dongjoon-hyun commented on pull request #28529: [SPARK-31692][SQL] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #28529:
URL: https://github.com/apache/spark/pull/28529#issuecomment-628797165


   Hmm. In this case, I believe we had better have the conf patch because we can disable it when we have a regression on this PR.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #28529: [SPARK-31692][SQL][2.4] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28529:
URL: https://github.com/apache/spark/pull/28529#issuecomment-629738100


   **[Test build #122748 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122748/testReport)** for PR 28529 at commit [`c96d6b0`](https://github.com/apache/spark/commit/c96d6b0e4bd2effbf8ddc2f2d7acd0890dae168a).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #28529: [SPARK-31692][SQL] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28529:
URL: https://github.com/apache/spark/pull/28529#issuecomment-628772181


   Can one of the admins verify this patch?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #28529: [SPARK-31692][SQL] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28529:
URL: https://github.com/apache/spark/pull/28529#issuecomment-628783312






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #28529: [SPARK-31692][SQL] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28529:
URL: https://github.com/apache/spark/pull/28529#issuecomment-628783171






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] dongjoon-hyun commented on pull request #28529: [SPARK-31692][SQL] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #28529:
URL: https://github.com/apache/spark/pull/28529#issuecomment-628780364


   ok to test


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #28529: [SPARK-31692][SQL] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28529:
URL: https://github.com/apache/spark/pull/28529#issuecomment-628783171






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] karuppayya commented on pull request #28529: [SPARK-31692][SQL] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
karuppayya commented on pull request #28529:
URL: https://github.com/apache/spark/pull/28529#issuecomment-628767171


   Backport commit for SPARK-31692 @dongjoon-hyun 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] dongjoon-hyun edited a comment on pull request #28529: [SPARK-31692][SQL] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on pull request #28529:
URL: https://github.com/apache/spark/pull/28529#issuecomment-628797165


   Hmm. In this case, I believe we had better have the conf patch because we can disable it when we have a regression on this PR. Let me take a look at that.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #28529: [SPARK-31692][SQL] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28529:
URL: https://github.com/apache/spark/pull/28529#issuecomment-628771209


   Can one of the admins verify this patch?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #28529: [SPARK-31692][SQL][2.4] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28529:
URL: https://github.com/apache/spark/pull/28529#issuecomment-629718619






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] dongjoon-hyun commented on pull request #28529: [SPARK-31692][SQL][2.4] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #28529:
URL: https://github.com/apache/spark/pull/28529#issuecomment-629718381


   Retest this please


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #28529: [SPARK-31692][SQL] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28529:
URL: https://github.com/apache/spark/pull/28529#issuecomment-628772181


   Can one of the admins verify this patch?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #28529: [SPARK-31692][SQL] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28529:
URL: https://github.com/apache/spark/pull/28529#issuecomment-628783321


   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122625/
   Test FAILed.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28529: [SPARK-31692][SQL] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #28529:
URL: https://github.com/apache/spark/pull/28529#discussion_r425313718



##########
File path: sql/core/src/main/scala/org/apache/spark/sql/internal/SharedState.scala
##########
@@ -157,7 +157,13 @@ private[sql] class SharedState(val sparkContext: SparkContext) extends Logging {
 
 object SharedState extends Logging {
   try {
-    URL.setURLStreamHandlerFactory(new FsUrlStreamHandlerFactory())
+    SparkSession.getActiveSession match {

Review comment:
       Is this because we don't have a configuration patch in branch-2.4?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #28529: [SPARK-31692][SQL][2.4] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28529:
URL: https://github.com/apache/spark/pull/28529#issuecomment-629718619






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28529: [SPARK-31692][SQL] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #28529:
URL: https://github.com/apache/spark/pull/28529#discussion_r425312918



##########
File path: sql/core/src/main/scala/org/apache/spark/sql/internal/SharedState.scala
##########
@@ -157,7 +157,13 @@ private[sql] class SharedState(val sparkContext: SparkContext) extends Logging {
 
 object SharedState extends Logging {
   try {
-    URL.setURLStreamHandlerFactory(new FsUrlStreamHandlerFactory())
+    SparkSession.getActiveSession match {

Review comment:
       Ur, is it the same with branch-3.0?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #28529: [SPARK-31692][SQL] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28529:
URL: https://github.com/apache/spark/pull/28529#issuecomment-628846232






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] dongjoon-hyun closed pull request #28529: [SPARK-31692][SQL][2.4] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun closed pull request #28529:
URL: https://github.com/apache/spark/pull/28529


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] dongjoon-hyun commented on pull request #28529: [SPARK-31692][SQL] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #28529:
URL: https://github.com/apache/spark/pull/28529#issuecomment-628780445


   Thank you, @karuppayya !


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] dongjoon-hyun edited a comment on pull request #28529: [SPARK-31692][SQL] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on pull request #28529:
URL: https://github.com/apache/spark/pull/28529#issuecomment-628780445


   Thank you, @karuppayya ! I updated the PR description like the original PR.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #28529: [SPARK-31692][SQL] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28529:
URL: https://github.com/apache/spark/pull/28529#issuecomment-628771209


   Can one of the admins verify this patch?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #28529: [SPARK-31692][SQL][2.4] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28529:
URL: https://github.com/apache/spark/pull/28529#issuecomment-629718514


   **[Test build #122748 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122748/testReport)** for PR 28529 at commit [`c96d6b0`](https://github.com/apache/spark/commit/c96d6b0e4bd2effbf8ddc2f2d7acd0890dae168a).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #28529: [SPARK-31692][SQL][2.4] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28529:
URL: https://github.com/apache/spark/pull/28529#issuecomment-629738163






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] dongjoon-hyun commented on pull request #28529: [SPARK-31692][SQL][2.4] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #28529:
URL: https://github.com/apache/spark/pull/28529#issuecomment-630500856


   @karuppayya . I cherry-picked your original commit from `branch-3.0` to `branch-2.4` and resolved the conflicts. Thanks!


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] dongjoon-hyun commented on pull request #28529: [SPARK-31692][SQL][2.4] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #28529:
URL: https://github.com/apache/spark/pull/28529#issuecomment-630497452


   Thank you for your patience, @karuppayya .
   For this PR, I backported two commits additionally as a preparation.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #28529: [SPARK-31692][SQL] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28529:
URL: https://github.com/apache/spark/pull/28529#issuecomment-628782552


   **[Test build #122625 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122625/testReport)** for PR 28529 at commit [`e7b378f`](https://github.com/apache/spark/commit/e7b378f12a7bf0ad6fc35f3bc58d83856b824d07).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA removed a comment on pull request #28529: [SPARK-31692][SQL] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28529:
URL: https://github.com/apache/spark/pull/28529#issuecomment-628782552


   **[Test build #122625 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122625/testReport)** for PR 28529 at commit [`e7b378f`](https://github.com/apache/spark/commit/e7b378f12a7bf0ad6fc35f3bc58d83856b824d07).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #28529: [SPARK-31692][SQL] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28529:
URL: https://github.com/apache/spark/pull/28529#issuecomment-628846232


   Merged build finished. Test FAILed.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #28529: [SPARK-31692][SQL][2.4] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28529:
URL: https://github.com/apache/spark/pull/28529#issuecomment-628846237


   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122627/
   Test FAILed.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #28529: [SPARK-31692][SQL][2.4] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28529:
URL: https://github.com/apache/spark/pull/28529#issuecomment-629738163






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #28529: [SPARK-31692][SQL] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28529:
URL: https://github.com/apache/spark/pull/28529#issuecomment-628789602


   **[Test build #122627 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122627/testReport)** for PR 28529 at commit [`c96d6b0`](https://github.com/apache/spark/commit/c96d6b0e4bd2effbf8ddc2f2d7acd0890dae168a).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA removed a comment on pull request #28529: [SPARK-31692][SQL][2.4] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28529:
URL: https://github.com/apache/spark/pull/28529#issuecomment-629718514


   **[Test build #122748 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122748/testReport)** for PR 28529 at commit [`c96d6b0`](https://github.com/apache/spark/commit/c96d6b0e4bd2effbf8ddc2f2d7acd0890dae168a).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #28529: [SPARK-31692][SQL] Pass hadoop confs specifed via Spark confs to URLStreamHandlerfactory

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28529:
URL: https://github.com/apache/spark/pull/28529#issuecomment-628783296


   **[Test build #122625 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122625/testReport)** for PR 28529 at commit [`e7b378f`](https://github.com/apache/spark/commit/e7b378f12a7bf0ad6fc35f3bc58d83856b824d07).
    * This patch **fails Scala style tests**.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org