[GitHub] HyukjinKwon opened a new pull request #23728: [SPARK-26818][ML] Make MLEvents JSON ser/de safe

[GitBox <gi...@apache.org>]

HyukjinKwon opened a new pull request #23728: [SPARK-26818][ML] Make MLEvents JSON ser/de safe
URL: https://github.com/apache/spark/pull/23728
 
 
   ## What changes were proposed in this pull request?
   
   Currently, it looks it's not going to cause any virtually effective problem apparently (if I didn't misread the codes).
   
   I see one place that JSON formatted events are being used.
   
   https://github.com/apache/spark/blob/ec506bd30c2ca324c12c9ec811764081c2eb8c42/core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala#L148
   
    because it just logs when the exception is ignorable  
   
   https://github.com/apache/spark/blob/9690eba16efe6d25261934d8b73a221972b684f3/core/src/main/scala/org/apache/spark/util/ListenerBus.scala#L111
   
   Yes, it looks I can explicitly disable `logEvent` in `SparkListenerEvent`  as well. However, I guess it should be best to stay safe - I don't want this unstable experimental feature breaks anything in any case.
   
   This is also to match SQL execution events side:
   
   https://github.com/apache/spark/blob/ca545f79410a464ef24e3986fac225f53bb2ef02/sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLListener.scala#L41-L57
   
   to make ML events JSON ser/de safe.
   
   ## How was this patch tested?
   
   Manually tested, and unit tests were added.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org