You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@spark.apache.org by Daniel Templeton <dt...@twitter.com.INVALID> on 2020/04/21 20:57:24 UTC

ATSv2 Integration

Hi!  I see that SPARK-1537 is now closed as won't fix.  What was the 
reason the Spark community decided to skip ATSv2 support?

My org is currently looking at how we introduce Spark into YARN clusters 
that are primarily used for Scalding today.  For the Scalding UI we use 
an internal tool that's backed by ATSv2.  It seems like it would make 
good sense for us to have Spark also storing metrics in ATSv2, rather 
than maintaining two separate metrics stores, which would also make it 
easier for us to have a single common UI.

If we were willing to take up the work to integrate Spark with ATSv2, 
would the community be willing to support that effort?

Thanks!
Daniel

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org