certification suite?

[William Benton <wi...@redhat.com>]

Hi all,

Does anyone happen to know what tests Databricks uses for the Spark
distribution certification suite?  Is it simply the tests that run as CI on
Spark pull requests, or is there something more involved?

The web site (
https://databricks.com/spark/certification/certified-spark-distribution)
says that "the certification process is fully transparent with open-source
tests (developed and maintained by the community and used to test each
release of Apache Spark), lightweight, and 100% free."  But that's all the
information I can find about the process, so I was hoping someone could
elucidate it for me.

It seems like community-based downstream distributions (e.g., BigTop,
Fedora, etc) could benefit from running a similar test suite to the
official Spark distribution certification suite in their regular CI, even
if the results were (necessarily) not official.


best,
wb