[jira] [Commented] (SOLR-13722) A cluster-wide blob upload package option & avoid remote url

["David Smiley (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-13722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16934320#comment-16934320 ] 

David Smiley commented on SOLR-13722:
-------------------------------------

If I'm reading this correctly, this issue is purely a refactoring to change Solr's existing "blob" store to be a some new local disk replicated thing instead of a collection.  True?  If so, as nice as this is, this seems lesser priority than other items of the umbrella ticket since what we have now works.  I'm not trying to say we shouldn't do it but just want to validate my understanding of the matter.  And If I'm right, this sort of thing could come much later as an improvement to our plugin story.

> A cluster-wide blob upload package option & avoid remote url
> ------------------------------------------------------------
>
>                 Key: SOLR-13722
>                 URL: https://issues.apache.org/jira/browse/SOLR-13722
>             Project: Solr
>          Issue Type: Sub-task
>      Security Level: Public(Default Security Level. Issues are Public) 
>            Reporter: Noble Paul
>            Assignee: Noble Paul
>            Priority: Major
>              Labels: package
>
> This ticket totally eliminates the need for an external service to host the jars. So a url will no longer be required. An external URL leads to unreliability because the service may go offline or it can be DDoSed if/when too many requests are sent to them
>  
>  
>  Add a jar to cluster as follows
> {code:java}
> curl -X POST -H 'Content-Type: application/octet-stream' --data-binary @myjar.jar http://localhost:8983/api/cluster/blob
> {code}
> This does the following operations
>  * Upload this jar to all the live nodes in the system
>  * The name of the file is the {{sha256}} of the file/payload
>  * The blob is agnostic of the content of the file/payload
> h2.  How it works?
> A blob that is POSTed to the {{/api/cluster/blob}} end point is persisted locally & all nodes are instructed to download it from this node or from any other available node. If a node comes up later, it can query other nodes in the system and download the blobs as required
> h2. {{add-package}} command
> {code:java}
> curl -X POST -H 'Content-type:application/json' --data-binary '{
>   "add-package": {
>        "name": "my-package" ,
>       "sha256":"<the-sha256>"
>   }}' http://localhost:8983/api/cluster
> {code}
>  The {{sha256}} is the same as the file name. It gets hold of the jar using the following steps
>  * check the local file system for the blob
>  * If not available locally,  query other live nodes if they have the blob (one by one)
>  * if a node has it , it's downloaded and persisted to it's local {{blob}} dir
> h2. Security
> The blob upload does not check for the content of the payload and it does not verify the file. However, the {{add-package}} , {{update-package}} commands check for the signatures (if enabled) . 
>  The size of the file is limited to 5MB,to avoid (OOM). This can be changed using a system property {{runtime.lib.size}} . 
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org