Spark 3.4 Deterministic Simulation Testing (Bug Hunt Hackathon)

[Craig Alfieri <cr...@antithesis.com>]

Hi Spark User Group – In support of Spark’s community/OSS project, my company Antithesis, is holding a hackathon in a couple of weeks focusing on finding bug conditions in popular OSS projects.

One of the teams participating is planning to work on Spark 3.4

We plan to run Spark 3.4 through a deterministic network simulator, while introducing elements of chaos via guided fuzzing to a multiple cluster set-up.

Where we can use guidance/support from the group, if there is a bug that continues to resurface, that could benefit from this approach, we’d love to help solve for this in both support of this community and our hackathon.

Please reach out if there are any suggestions where our energy/efforts can be best focused.


[signature_901675016]
Craig Alfieri
c: 917.841.1652
Craig.Alfieri@antithesis.com<ma...@antithesis.com>
New York, NY.
Antithesis.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.antithesis.com_&d=DwMFaQ&c=7563p3e2zaQw0AB1wrFVgyagb2IE5rTZOYPxLxfZlX4&r=1FbSpGgVIpZO4QkQDmXk7jc1BFVciZWVioOvdJ86ubY&m=5SVjNvtYuy6icWSaP0lwjzTQw1Cc7JQO9QVaxn5KxqTdH8HC1HHURutlp5rgiaMH&s=SRmgBE5ImnGZ-GuqL3X6Q_6NPYiay1gLRbcUUofPIHo&e=>

We can't talk about most of the bugs that we've found for our customers,
but some customers like to speak about their work with us:
https://github.com/mongodb/mongo/wiki/Testing-MongoDB-with-Antithesis



-- 

*-----------------------------*
*This email and any files transmitted with 
it are confidential and intended solely for the use of the individual or 
entity for whom they are addressed. If you received this message in error, 
please notify the sender and remove it from your system.*