[jira] [Commented] (SOLR-16525) Periodic performance benchmarking

["Ishan Chattopadhyaya (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-16525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17654032#comment-17654032 ] 

Ishan Chattopadhyaya commented on SOLR-16525:
---------------------------------------------

Thanks for your questions, Jason.

bq. Is there any relationship between the solr-bench code, and the JMH-based benchmarking module that Mark added recently? e.g. Do they overlap functionally, or are they targeting different aspects of performance? Are there guidelines on when folks should write one type of benchmark vs the other?

No overlap. JMH is for micro-benchmarking of parts of Solr, solr-bench is for benchmarking and stress testing for end to end functionality. It would be nice to integrate the benchmarking module in the solr-bench reports.

bq. What's the relationship between solr-bench and the Solr community? I see solr-bench came from a GSoC project run within the community, but I guess it never got merged at the time and only recently-ish (2020) re-emerged under the "fullstory" (and "SearchScale") orgs in Github: is the eventual goal that this would live in solr-sandbox or somewhere else one day, and be ASF/community-maintained code?

The GSoC project was furthered by FullStory into solr-bench. We are trying to use it for periodic benchmarks for the community.

bq. In terms of hosting, and running, and hardware and all that: is solr-bench something that could conceivably eventually run on an ASF Jenkins (say, nightly), or would that be prohibitive for cost/hardware or other reasons, or is that one of the open questions at this point?

As of now, it is running on hardware donated by SearchScale & FullStory. We plan to add more servers to it. ASF Jenkins hardware would be nice, but whoever maintains it would need to keep it same and do periodic maintanance. It is hence very much an open question.

> Periodic performance benchmarking
> ---------------------------------
>
>                 Key: SOLR-16525
>                 URL: https://issues.apache.org/jira/browse/SOLR-16525
>             Project: Solr
>          Issue Type: Task
>      Security Level: Public(Default Security Level. Issues are Public) 
>            Reporter: Ishan Chattopadhyaya
>            Priority: Major
>         Attachments: image-2022-11-09-13-23-21-131.png, periodic-benchmarks.odp
>
>
> This issue is about periodic performance testing of Solr commits against pre-written suites of performance tests.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org