[GitHub] [lucene-solr] thelabdude edited a comment on pull request #1845: SOLR-14613: Autoscaling replacement using placement plugins

[GitBox <gi...@apache.org>]

thelabdude edited a comment on pull request #1845:
URL: https://github.com/apache/lucene-solr/pull/1845#issuecomment-690613460


   Building on @noblepaul 's concern, I do think we need to reconcile the newly added interfaces in the `org.apache.solr.cluster.api` package (under solrj) with the very similar interfaces in the `org.apache.solr.cluster.placement` package added in this PR. I do realize the `org.apache.solr.cluster.api` package was introduced while the work on this PR was already underway. My intent here is to move forward with a plan we’re all comfortable with and not rehash the past.
   
   To recap my understanding from the previous PR comments, the intent of having the Node, SolrCollection, Shard, Replica interfaces in `org.apache.solr.cluster.placement` is to avoid exposing SolrCloud internals to the plugin writer.
   
   While I agree that is a laudable goal in general esp. for plugins, this particular framework actually needs to have access to internal information about the cluster. In other words, placing replicas is a very internal (and core) concern. The fact we're exposing this as a pluggable implementation is really for operational convenience. 
   
   Moreover, I do believe implementing a placement strategy requires most of the metadata present in collections, shards, replicas, and nodes, so I don’t know if the cost of having two representations of the same domain objects in two different places is worth the benefit it provides? I think the community needs to decide this is how we want to move forward.
   
   So referring back to Ilan's stated goals:
   1. placement plugin writing is easy, 
   2. implementation for this placement API is efficient, and 
   3. plugins do not break when Solr is refactored
   
   I’d actually argue that goal `#1` is subjective and hard to measure. For instance, do we consider the SamplePluginAffinityReplicaPlacement impl easy to write? It seems to require a fair amount of knowledge of how Solr’s various objects interact.
   
   I believe @murblanc  has done a great job with goal `#2` in this PR.
   
   For goal `#3`, my sense is that any refactoring probably cannot break the interfaces defined in `org.apache.solr.cluster.api` without fundamentally changing the architecture of SolrCloud, which would likely break assumptions made in the `org.apache.solr.cluster.placement` as well.
   
   Lastly, from our slack conversation, I was only suggesting that instead of the plugin impl introducing the `AzWithNodes` that we simply formalize this as a `DataCenter` in the API, which basically does what `AzWithNodes` is doing in the code, not a big deal though ...
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org