[GitHub] spark issue #21376: [SPARK-24250][SQL] support accessing SQLConf inside task...

[squito <gi...@git.apache.org>]

Github user squito commented on the issue:

    https://github.com/apache/spark/pull/21376
  
    I'm still seeing a lot of build failures which seem to be related to this (accessing a conf in a task in turn accesses the LiveListenerBus).  Is this something new?  Or related to this change?  eg.
    https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4193/testReport/org.apache.spark.sql.execution/UnsafeRowSerializerSuite/toUnsafeRow___test_helper_method/
    
    ```
    sbt.ForkMain$ForkError: java.lang.IllegalStateException: LiveListenerBus is stopped.
    	at org.apache.spark.scheduler.LiveListenerBus.addToQueue(LiveListenerBus.scala:97)
    	at org.apache.spark.scheduler.LiveListenerBus.addToStatusQueue(LiveListenerBus.scala:80)
    	at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:93)
    	at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:120)
    	at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:120)
    	at scala.Option.getOrElse(Option.scala:121)
    	at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:120)
    	at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:119)
    	at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:286)
    	at org.apache.spark.sql.test.TestSparkSession.sessionState$lzycompute(TestSQLContext.scala:42)
    	at org.apache.spark.sql.test.TestSparkSession.sessionState(TestSQLContext.scala:41)
    	at org.apache.spark.sql.SparkSession$$anonfun$1$$anonfun$apply$1.apply(SparkSession.scala:95)
    	at org.apache.spark.sql.SparkSession$$anonfun$1$$anonfun$apply$1.apply(SparkSession.scala:95)
    	at scala.Option.map(Option.scala:146)
    	at org.apache.spark.sql.SparkSession$$anonfun$1.apply(SparkSession.scala:95)
    	at org.apache.spark.sql.SparkSession$$anonfun$1.apply(SparkSession.scala:94)
    	at org.apache.spark.sql.internal.SQLConf$.get(SQLConf.scala:126)
    	at org.apache.spark.sql.catalyst.expressions.CodeGeneratorWithInterpretedFallback.createObject(CodeGeneratorWithInterpretedFallback.scala:54)
    	at org.apache.spark.sql.catalyst.expressions.UnsafeProjection$.create(Projection.scala:157)
    	at org.apache.spark.sql.catalyst.expressions.UnsafeProjection$.create(Projection.scala:150)
    	at org.apache.spark.sql.execution.UnsafeRowSerializerSuite.org$apache$spark$sql$execution$UnsafeRowSerializerSuite$$unsafeRowConverter(UnsafeRowSerializerSuite.scala:54)
    	at org.apache.spark.sql.execution.UnsafeRowSerializerSuite.org$apache$spark$sql$execution$UnsafeRowSerializerSuite$$toUnsafeRow(UnsafeRowSerializerSuite.scala:49)
    	at org.apache.spark.sql.execution.UnsafeRowSerializerSuite$$anonfun$2.apply(UnsafeRowSerializerSuite.scala:63)
    	at org.apache.spark.sql.execution.UnsafeRowSerializerSuite$$anonfun$2.apply(UnsafeRowSerializerSuite.scala:60)
    ...
    ```


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org