You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@solr.apache.org by "anilkhichar (via GitHub)" <gi...@apache.org> on 2023/07/20 14:27:55 UTC

[GitHub] [solr-operator] anilkhichar opened a new issue, #593: Could not find a backup repository with name `solr-backup-s3-repo`

anilkhichar opened a new issue, #593:
URL: https://github.com/apache/solr-operator/issues/593

   Please suggest any pointers for debugging. 
   
   Is there any API where we can get the list of SolrOperator registered backup repositories?
   Also, for S3 backup, does the Operator first download the backup data on disk and then execute the S3 cp/sync? 
   
   Any internal detail would be helpful to debug the core issue.
   
   **Error:**
   ```
   2023-07-18T23:18:54.905Z        
   ERROR   controller-runtime.manager.controller.solrbackup        
   Error while taking SolrCloud backup     
   {"reconciler group": "solr.apache.org", 
   "reconciler kind": "SolrBackup",
    "name": "solr-backup", 
   "namespace": "ns-solrcloud-int", 
   "error": "Received bad response code of 500 from solr with response: 
   {\n  \"responseHeader\":
   {\n    \"status\":500,\n    \"QTime\":0},\n  \"error\":{\n    
   \"msg\":\"Could not find a backup repository with name solr-backup-s3-repo\",
   \n    \"trace\":\"java.lang.NullPointerException: 
   Could not find a backup repository with name solr-backup-s3-repo\\
   n\\tat 
   java.base/java.util.Objects.requireNonNull(Unknown Source)\\n\\
   tat org.apache.solr.core.backup.repository.
   BackupRepositoryFactory.newInstance(BackupRepositoryFactory.java:74)
   ```
   
   
   - **Solrbackup Setup**
   `kubectl edit solrcloud solrsetup`
   
     ```
     backupRepositories:
     - name: "solr-backup-s3-repo"
       s3:
         region: "us-west-2"
         bucket: "<solr-s3-bucket>"
     ```
   
   - **SolrBackup**
     ```
     ---
     apiVersion: solr.apache.org/v1beta1
     kind: SolrBackup
     metadata:
       name: "solr-backup"
     spec:
       repositoryName: "solr-backup-s3-repo"
       solrCloud: "solrsetup"
       collections:
         - mycollection
     ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org

[GitHub] [solr-operator] anilkhichar commented on issue #593: Could not find a backup repository with name `solr-backup-s3-repo`

Posted by "anilkhichar (via GitHub)" <gi...@apache.org>.

anilkhichar commented on issue #593:
URL: https://github.com/apache/solr-operator/issues/593#issuecomment-1644229419

   Thank you @HoustonPutman for looking into this.
   
   We are using Solr 9.2.0 with SolrOperator 0.5.1.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org

[GitHub] [solr-operator] sbbagal13 commented on issue #593: Could not find a backup repository with name `solr-backup-s3-repo`

Posted by "sbbagal13 (via GitHub)" <gi...@apache.org>.

sbbagal13 commented on issue #593:
URL: https://github.com/apache/solr-operator/issues/593#issuecomment-1656422604

   Thanks @HoustonPutman - I am using MINIO as storage.. I just found this [Solr website](https://solr.apache.org/guide/8_11/making-and-restoring-backups.html)  
   **You can use the s3.endpoint option to use this BackupRepository with s3-compatible endpoints. Beware that not all s3-compatible endpoints will work with the S3BackupRepository. Minio is an example of an s3-compatible endpoint that does not work with the S3BackupRepository. The S3BackupRepository is only guaranteed to be compatible with AWS S3 and S3Mock.**
   Is this valid with Solr-operator ? 
   
   Here are both the YAMLs
   **SOLRCLOUD**
   
   `apiVersion: solr.apache.org/v1beta1
   kind: SolrCloud
   metadata:
     name: cstone-solr2
   spec:
     dataStorage:
       persistent:
         reclaimPolicy: Retain
         pvcTemplate:
           spec:
             resources:
               requests:
                 storage: "20Gi"
     replicas: 3
     solrImage:
       repository: myregistry/images/solr
       tag: 8.11.1
       imagePullSecret: "default-gitlab-registry"
       pullPolicy: "Always"
   
     solrJavaMem: "-Xms1g -Xmx3g"
     customSolrKubeOptions:
       podOptions:
         livenessProbe:
           initialDelaySeconds: 20
           timeoutSeconds: 50
           periodSeconds: 10
           successThreshold: 1
           failureThreshold: 3
         readinessProbe:
           initialDelaySeconds: 20
           timeoutSeconds: 50
           periodSeconds: 10
           successThreshold: 1
           failureThreshold: 3
   
         podSecurityContext: {}
         #  runAsUser: 1000890000
         resources:
           limits:
             memory: "8G"
           requests:
             cpu: "500m"
             memory: "4G"
     zookeeperRef:
       provided:
         persistence:
           reclaimPolicy: "Retain"
           spec:
             resources:
               requests:
                 storage: "20Gi"
   
     solrOpts: "-Dsolr.autoSoftCommit.maxTime=10000"
     solrGCTune: "-XX:SurvivorRatio=4 -XX:TargetSurvivorRatio=90 -XX:MaxTenuringThreshold=8"
     backupRepositories:
       - name: "s3-backups-1"
         s3:
           region: "us-east-1" # Required
           bucket: "cstonebackup" # Required
           credentials:
             accessKeyIdSecret: # Optional
               name: minio-cstonedfs
               key: AWS_ACCESS_KEY_ID
             secretAccessKeySecret: # Optional
               name: minio-cstonedfs
               key: AWS_SECRET_ACCESS_KEY
           endpoint: "http://minio-endpoint-URL:80" # `
           
           **BACKUP** 
           
           `apiVersion: solr.apache.org/v1beta1
   kind: SolrBackup
   metadata:
     name: solr-backup
   spec:
     repositoryName: "s3-backups-1"
     solrCloud: cstone-solr2
     location: "cstone-solr2"`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org

[GitHub] [solr-operator] anilkhichar commented on issue #593: Could not find a backup repository with name `solr-backup-s3-repo`

Posted by "anilkhichar (via GitHub)" <gi...@apache.org>.

anilkhichar commented on issue #593:
URL: https://github.com/apache/solr-operator/issues/593#issuecomment-1648850445

   Thank you @HoustonPutman for the guidance. Yes, we are using custom solr xml and that was the root cause. I wish, the solr backup document would have mentioned it. I will raise PR for that separately.
   
   **The Resolution:**
   1. Configure S3 jars/lib path in the sharedPath
   1. Configure S3 repository in the custom solr xml
       ```
       <?xml version="1.0" encoding="UTF-8" ?>
       <solr>
         <str name="sharedLib">${solr.sharedLib:},${solr.install.dir}/contrib/s3-repository/lib</str>
         <solrcloud>
         .....
         </solrcloud>
         <backup>
           <repository name="solr-backup-s3-repo" class="org.apache.solr.s3.S3BackupRepository">
             <str name="s3.bucket.name">solr-backup-s3-bucket</str>
             <str name="s3.region">us-east-1</str>
           </repository>
         </backup>
       </solr>
       ```
   1. Then on top it, it’s regular Solr Operator backup/restore configurations:
       ```
       kubectl edit solrcloud my-solr-cloud
    
       solrBackup:
         enabled: "true"
         backupRepositories:
           name: "solr-backup-s3-repo"
           region: "us-east-1"
           bucket: "solr-backup-s3-bucket" 
       ```
   1. Then on top it, it’s regular Solr Operator backup/restore configurations:
       ```
       apiVersion: solr.apache.org/v1beta1
       kind: SolrBackup
       metadata:
         name: dev-solr-backup
         namespace: solr-workload-namespace
       spec: 
         repositoryName: "solr-backup-s3-repo"
         solrCloud: "my-solr-cloud"
         collections: # upgraded solr operator versions can backup all available collections
           - my-collection
         recurrence: # Store one backup daily, and keep a week at a time.
           schedule: "@daily"
           maxSaved: 7
           disabled: false
       ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org