[jira] [Updated] (SOLR-13722) Package Management APIs

["Noble Paul (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SOLR-13722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Noble Paul updated SOLR-13722:
------------------------------
    Description: 
This ticket totally eliminates the need for an external service to host the jars. So a url will no longer be required. An external URL leads to unreliability because the service may go offline or it can be DDoSed if/when too many requests are sent to them

 

 
 Add a jar to cluster as follows
{code:java}
curl -X POST -H 'Content-Type: application/octet-stream' --data-binary @myjar.jar http://localhost:8983/api/cluster/filestore/package?name=myjar.jar
{code}
This does the following operations
 * Upload this jar to all the live nodes in the system
 * The name of the file is the {{sha256-<name>}} of the file/payload
 * The store is agnostic of the content of the file/payload

h2.  How it works?

A blob that is POSTed to the {{/api/cluster/blob}} end point is persisted locally & all nodes are instructed to download it from this node or from any other available node. If a node comes up later, it can query other nodes in the system and download the blobs as required
h2. {{add}} package command
{code:java}
curl -X POST -H 'Content-type:application/json' --data-binary '{
  "add": {
       "name": "my-package" ,
      "file":{"id" : "<the-sha256>", "sig" : "<signature>"}
  }}' http://localhost:8983/api/cluster/package
{code}
]


 

 

  was:
This ticket totally eliminates the need for an external service to host the jars. So a url will no longer be required. An external URL leads to unreliability because the service may go offline or it can be DDoSed if/when too many requests are sent to them

 

 
 Add a jar to cluster as follows
{code:java}
curl -X POST -H 'Content-Type: application/octet-stream' --data-binary @myjar.jar http://localhost:8983/api/cluster/blob
{code}
This does the following operations
 * Upload this jar to all the live nodes in the system
 * The name of the file is the {{sha256}} of the file/payload
 * The blob is agnostic of the content of the file/payload

h2.  How it works?

A blob that is POSTed to the {{/api/cluster/blob}} end point is persisted locally & all nodes are instructed to download it from this node or from any other available node. If a node comes up later, it can query other nodes in the system and download the blobs as required
h2. {{add-package}} command
{code:java}
curl -X POST -H 'Content-type:application/json' --data-binary '{
  "add-package": {
       "name": "my-package" ,
      "sha256":"<the-sha256>"
  }}' http://localhost:8983/api/cluster
{code}
 The {{sha256}} is the same as the file name. It gets hold of the jar using the following steps
 * check the local file system for the blob
 * If not available locally,  query other live nodes if they have the blob (one by one)
 * if a node has it , it's downloaded and persisted to it's local {{blob}} dir

h2. Security

The blob upload does not check for the content of the payload and it does not verify the file. However, the {{add-package}} , {{update-package}} commands check for the signatures (if enabled) . 
 The size of the file is limited to 5MB,to avoid (OOM). This can be changed using a system property {{runtime.lib.size}} . 

 

 


> Package Management APIs
> -----------------------
>
>                 Key: SOLR-13722
>                 URL: https://issues.apache.org/jira/browse/SOLR-13722
>             Project: Solr
>          Issue Type: Sub-task
>      Security Level: Public(Default Security Level. Issues are Public) 
>            Reporter: Noble Paul
>            Assignee: Noble Paul
>            Priority: Major
>              Labels: package
>
> This ticket totally eliminates the need for an external service to host the jars. So a url will no longer be required. An external URL leads to unreliability because the service may go offline or it can be DDoSed if/when too many requests are sent to them
>  
>  
>  Add a jar to cluster as follows
> {code:java}
> curl -X POST -H 'Content-Type: application/octet-stream' --data-binary @myjar.jar http://localhost:8983/api/cluster/filestore/package?name=myjar.jar
> {code}
> This does the following operations
>  * Upload this jar to all the live nodes in the system
>  * The name of the file is the {{sha256-<name>}} of the file/payload
>  * The store is agnostic of the content of the file/payload
> h2.  How it works?
> A blob that is POSTed to the {{/api/cluster/blob}} end point is persisted locally & all nodes are instructed to download it from this node or from any other available node. If a node comes up later, it can query other nodes in the system and download the blobs as required
> h2. {{add}} package command
> {code:java}
> curl -X POST -H 'Content-type:application/json' --data-binary '{
>   "add": {
>        "name": "my-package" ,
>       "file":{"id" : "<the-sha256>", "sig" : "<signature>"}
>   }}' http://localhost:8983/api/cluster/package
> {code}
> ]
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org