Job Submission Limitation

[Aaron Householder <aa...@aucsolutions.com>]

Hi Airavata Team,

I hope you're doing well. I was running some tests to gather metrics over the weekend and realized I might have unintentionally load tested Airavata. Using a script, I was able to submit a number of jobs in quick succession. Everything worked smoothly up to 100 jobs, but when I increased it to 200 jobs submitted to the metascheduler within a minute, I started encountering issues.

I understand that this kind of job submission rate isn’t common, but I’m curious about what might be causing the limitation. Is this a design constraint within Airavata or is there something we could adjust on our end to noticeably improve scalability and performance?

Regards,
Aaron