You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@solr.apache.org by GitBox <gi...@apache.org> on 2021/10/28 21:49:37 UTC

[GitHub] [solr-operator] HoustonPutman opened a new pull request #359: Add ability to schedule recurring incremental backups

HoustonPutman opened a new pull request #359:
URL: https://github.com/apache/solr-operator/pull/359


   Resolves #303 
   
   Still needs a bit of love, but the bulk of the work should be there.
   
   TODO:
   
   - [ ] Documentation & Changelog entry
   - [ ] Cleaning up logic around restarts
   - [ ] Rebase after other backup PRs are merged
   - [ ] Manual testing


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr-operator] gerlowskija commented on pull request #359: Add ability to schedule recurring incremental backups

Posted by GitBox <gi...@apache.org>.
gerlowskija commented on pull request #359:
URL: https://github.com/apache/solr-operator/pull/359#issuecomment-957795580


   > now I realize you mean a `purgeEvery` option in the CRD. I think we can assume that would be equivalent to saveEvery.
   
   Yep, that's what I was suggesting - an option in the CRD to expose the option that's already available in Solr.  That SOLR-15703 you pointed to looks very serious 😬 .  I'll take a look and hopefully get the Solr concern out of the way so we can consider exposure in the operator.
   
   When you mention "saveEvery" I assume you mean maxSaved?  Or is there some other "saveEvery" param I missed?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr-operator] gerlowskija commented on a change in pull request #359: Add ability to schedule recurring incremental backups

Posted by GitBox <gi...@apache.org>.
gerlowskija commented on a change in pull request #359:
URL: https://github.com/apache/solr-operator/pull/359#discussion_r740493719



##########
File path: api/v1beta1/solrbackup_types.go
##########
@@ -64,6 +71,29 @@ func (spec *SolrBackupSpec) withDefaults(backupName string) (changed bool) {
 	return changed
 }
 
+// BackupRecurrence defines the recurrence of the incremental backup
+type BackupRecurrence struct {
+	// Perform a backup on the given schedule, in CRON format.
+	//
+	// Multiple CRON syntaxes are supported
+	//   - Standard CRON (e.g. "CRON_TZ=Asia/Seoul 0 6 * * ?")
+	//   - Predefined Schedules (e.g. "@yearly", "@weekly", "@daily", etc.)
+	//   - Intervals (e.g. "@every 10h30m")
+	//
+	// For more information please check this reference:
+	// https://pkg.go.dev/github.com/robfig/cron/v3?utm_source=godoc#hdr-CRON_Expression_Format
+	Schedule string `json:"schedule"`
+
+	// Define the number of backup points to save for this backup at any given time.
+	// The oldest backups will be deleted if too many exist when a backup is taken.
+	// If not provided, this defaults to 10.
+	//
+	// +kubebuilder:default:=10
+	// +kubebuilder:validation:Minimum:=1
+	// +optional
+	MaxSaved int `json:"maxSaved,omitempty"`

Review comment:
       [0] This is mostly subjective preference, but I'd vote for a smaller default value here.  For collections with a lot of churn, 10 backups might be a lot of data to store by default.  Would you be open to a lower default like 3 or 5?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr-operator] gerlowskija edited a comment on pull request #359: Add ability to schedule recurring incremental backups

Posted by GitBox <gi...@apache.org>.
gerlowskija edited a comment on pull request #359:
URL: https://github.com/apache/solr-operator/pull/359#issuecomment-957795580


   > now I realize you mean a `purgeEvery` option in the CRD. I think we can assume that would be equivalent to saveEvery.
   
   Yep, that's what I was suggesting - an option in the CRD to expose the option that's already available in Solr.  That SOLR-15703 you pointed to looks very serious 😬 .  I'll take a look and hopefully get the Solr concern out of the way so we can consider exposure in the operator.
   
   When you mention "saveEvery" I assume you mean maxSaved?  Or is there some other "saveEvery" param I missed?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr-operator] HoustonPutman commented on pull request #359: Add ability to schedule recurring incremental backups

Posted by GitBox <gi...@apache.org>.
HoustonPutman commented on pull request #359:
URL: https://github.com/apache/solr-operator/pull/359#issuecomment-957772393






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr-operator] gerlowskija edited a comment on pull request #359: Add ability to schedule recurring incremental backups

Posted by GitBox <gi...@apache.org>.
gerlowskija edited a comment on pull request #359:
URL: https://github.com/apache/solr-operator/pull/359#issuecomment-957795580


   > now I realize you mean a `purgeEvery` option in the CRD. I think we can assume that would be equivalent to saveEvery.
   
   Yep, that's what I was suggesting - an option in the CRD to expose the option that's already available in Solr.  That SOLR-15703 you pointed to looks very serious 😬 .  I'll take a look and hopefully get the Solr concern out of the way so we can consider exposure in the operator.
   
   When you mention "saveEvery" I assume you mean maxSaved?  Or is there some other "saveEvery" param I missed?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr-operator] gerlowskija commented on pull request #359: Add ability to schedule recurring incremental backups

Posted by GitBox <gi...@apache.org>.
gerlowskija commented on pull request #359:
URL: https://github.com/apache/solr-operator/pull/359#issuecomment-954931247


   Cool - that all makes sense to me, thanks for clarifying!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr-operator] HoustonPutman commented on pull request #359: Add ability to schedule recurring incremental backups

Posted by GitBox <gi...@apache.org>.
HoustonPutman commented on pull request #359:
URL: https://github.com/apache/solr-operator/pull/359#issuecomment-954922376


   I never really considered a separate CRD. Generally CRDs are a necessary evil, and therefore the less the better IMO. The reason why this fits really well into a single CRD, in my opinion, is that the options would be identical between the two, except the recurring backup has an additional setting for `recurrence`. Because of this, it makes a ton of since to have them share the CRD. Otherwise every option we add to the SolrBackup CRD, we would also have to add to the RecurringSolrBackup CRD.
   
   Think of a kubernetes Headless Service / regular Service (with clusterIP). They are pretty different in usage, much further apart than SolrBackup and RecurringSolrBackup would be. However they share a single resource type, because they share almost every option, and are used for the same reasons. Users can distinguish them easily enough by looking at the options. (described more below)
   
   A CRD & controller pair is complex, the less complexity we add to the project, the better.
   
   > A user might be surprised that some solrbackup resources represent one-offs that will never impact the cluster again, and others represent ongoing periodic work that'll continue indefinitely.
   
   We can fix this quite easily. When you run `kubectl get solrbackups`, the data it returns is not random, it is selected by us [here](https://github.com/apache/solr-operator/blob/v0.4.0/api/v1beta1/solrbackup_types.go#L289). We can add a column(s) here to make it much more clear to the user what the solrbackup resource is doing. Maybe we add "Schedule" and/or "Next Backup Time". That way when you run `kubectl get solrbackups`, it is very easy to tell which ones are one-off, and which ones are recurring.
   
   I also really like the idea of someone taking a backup and then realizing they need it to be done on a schedule. They wouldn't need to delete the SolrBackup resource or create something else. All they would have to do is edit their SolrBackup resource, adding the `recurrence` section. At that point, the controller would work fine and treat it as a recurring backup that has been taken once already.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr-operator] gerlowskija commented on pull request #359: Add ability to schedule recurring incremental backups

Posted by GitBox <gi...@apache.org>.
gerlowskija commented on pull request #359:
URL: https://github.com/apache/solr-operator/pull/359#issuecomment-957795580


   > now I realize you mean a `purgeEvery` option in the CRD. I think we can assume that would be equivalent to saveEvery.
   
   Yep, that's what I was suggesting - an option in the CRD to expose the option that's already available in Solr.  That SOLR-15703 you pointed to looks very serious 😬 .  I'll take a look and hopefully get the Solr concern out of the way so we can consider exposure in the operator.
   
   When you mention "saveEvery" I assume you mean maxSaved?  Or is there some other "saveEvery" param I missed?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr-operator] HoustonPutman commented on pull request #359: Add ability to schedule recurring incremental backups

Posted by GitBox <gi...@apache.org>.
HoustonPutman commented on pull request #359:
URL: https://github.com/apache/solr-operator/pull/359#issuecomment-964314521


   @gerlowskija almost forgot, but I added the `disabled` flag and it works really well!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr-operator] gerlowskija commented on pull request #359: Add ability to schedule recurring incremental backups

Posted by GitBox <gi...@apache.org>.
gerlowskija commented on pull request #359:
URL: https://github.com/apache/solr-operator/pull/359#issuecomment-957795580


   > now I realize you mean a `purgeEvery` option in the CRD. I think we can assume that would be equivalent to saveEvery.
   
   Yep, that's what I was suggesting - an option in the CRD to expose the option that's already available in Solr.  That SOLR-15703 you pointed to looks very serious 😬 .  I'll take a look and hopefully get the Solr concern out of the way so we can consider exposure in the operator.
   
   When you mention "saveEvery" I assume you mean maxSaved?  Or is there some other "saveEvery" param I missed?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr-operator] HoustonPutman commented on a change in pull request #359: Add ability to schedule recurring incremental backups

Posted by GitBox <gi...@apache.org>.
HoustonPutman commented on a change in pull request #359:
URL: https://github.com/apache/solr-operator/pull/359#discussion_r745723018



##########
File path: api/v1beta1/solrbackup_types.go
##########
@@ -64,6 +71,29 @@ func (spec *SolrBackupSpec) withDefaults(backupName string) (changed bool) {
 	return changed
 }
 
+// BackupRecurrence defines the recurrence of the incremental backup
+type BackupRecurrence struct {
+	// Perform a backup on the given schedule, in CRON format.
+	//
+	// Multiple CRON syntaxes are supported
+	//   - Standard CRON (e.g. "CRON_TZ=Asia/Seoul 0 6 * * ?")
+	//   - Predefined Schedules (e.g. "@yearly", "@weekly", "@daily", etc.)
+	//   - Intervals (e.g. "@every 10h30m")
+	//
+	// For more information please check this reference:
+	// https://pkg.go.dev/github.com/robfig/cron/v3?utm_source=godoc#hdr-CRON_Expression_Format
+	Schedule string `json:"schedule"`
+
+	// Define the number of backup points to save for this backup at any given time.
+	// The oldest backups will be deleted if too many exist when a backup is taken.
+	// If not provided, this defaults to 10.
+	//
+	// +kubebuilder:default:=10
+	// +kubebuilder:validation:Minimum:=1
+	// +optional
+	MaxSaved int `json:"maxSaved,omitempty"`

Review comment:
       I set the default to 5




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr-operator] HoustonPutman commented on pull request #359: Add ability to schedule recurring incremental backups

Posted by GitBox <gi...@apache.org>.
HoustonPutman commented on pull request #359:
URL: https://github.com/apache/solr-operator/pull/359#issuecomment-957772393






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr-operator] gerlowskija edited a comment on pull request #359: Add ability to schedule recurring incremental backups

Posted by GitBox <gi...@apache.org>.
gerlowskija edited a comment on pull request #359:
URL: https://github.com/apache/solr-operator/pull/359#issuecomment-957795580


   > now I realize you mean a `purgeEvery` option in the CRD. I think we can assume that would be equivalent to saveEvery.
   
   Yep, that's what I was suggesting - an option in the CRD to expose the option that's already available in Solr.  That SOLR-15703 you pointed to looks very serious 😬 .  I'll take a look and hopefully get the Solr concern out of the way so we can consider exposure in the operator.
   
   When you mention "saveEvery" I assume you mean maxSaved?  Or is there some other "saveEvery" param I missed?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr-operator] HoustonPutman commented on a change in pull request #359: Add ability to schedule recurring incremental backups

Posted by GitBox <gi...@apache.org>.
HoustonPutman commented on a change in pull request #359:
URL: https://github.com/apache/solr-operator/pull/359#discussion_r741153892



##########
File path: api/v1beta1/solrbackup_types.go
##########
@@ -64,6 +71,29 @@ func (spec *SolrBackupSpec) withDefaults(backupName string) (changed bool) {
 	return changed
 }
 
+// BackupRecurrence defines the recurrence of the incremental backup
+type BackupRecurrence struct {
+	// Perform a backup on the given schedule, in CRON format.
+	//
+	// Multiple CRON syntaxes are supported
+	//   - Standard CRON (e.g. "CRON_TZ=Asia/Seoul 0 6 * * ?")
+	//   - Predefined Schedules (e.g. "@yearly", "@weekly", "@daily", etc.)
+	//   - Intervals (e.g. "@every 10h30m")
+	//
+	// For more information please check this reference:
+	// https://pkg.go.dev/github.com/robfig/cron/v3?utm_source=godoc#hdr-CRON_Expression_Format
+	Schedule string `json:"schedule"`
+
+	// Define the number of backup points to save for this backup at any given time.
+	// The oldest backups will be deleted if too many exist when a backup is taken.
+	// If not provided, this defaults to 10.
+	//
+	// +kubebuilder:default:=10
+	// +kubebuilder:validation:Minimum:=1
+	// +optional
+	MaxSaved int `json:"maxSaved,omitempty"`

Review comment:
       Very open. I literally just chose a number here. I am very happy to differ to whatever you think it should be. Give the call and it is done.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr-operator] gerlowskija commented on a change in pull request #359: Add ability to schedule recurring incremental backups

Posted by GitBox <gi...@apache.org>.
gerlowskija commented on a change in pull request #359:
URL: https://github.com/apache/solr-operator/pull/359#discussion_r740493719



##########
File path: api/v1beta1/solrbackup_types.go
##########
@@ -64,6 +71,29 @@ func (spec *SolrBackupSpec) withDefaults(backupName string) (changed bool) {
 	return changed
 }
 
+// BackupRecurrence defines the recurrence of the incremental backup
+type BackupRecurrence struct {
+	// Perform a backup on the given schedule, in CRON format.
+	//
+	// Multiple CRON syntaxes are supported
+	//   - Standard CRON (e.g. "CRON_TZ=Asia/Seoul 0 6 * * ?")
+	//   - Predefined Schedules (e.g. "@yearly", "@weekly", "@daily", etc.)
+	//   - Intervals (e.g. "@every 10h30m")
+	//
+	// For more information please check this reference:
+	// https://pkg.go.dev/github.com/robfig/cron/v3?utm_source=godoc#hdr-CRON_Expression_Format
+	Schedule string `json:"schedule"`
+
+	// Define the number of backup points to save for this backup at any given time.
+	// The oldest backups will be deleted if too many exist when a backup is taken.
+	// If not provided, this defaults to 10.
+	//
+	// +kubebuilder:default:=10
+	// +kubebuilder:validation:Minimum:=1
+	// +optional
+	MaxSaved int `json:"maxSaved,omitempty"`

Review comment:
       [0] This is mostly subjective preference, but I'd vote for a smaller default value here.  For collections with a lot of churn, 10 backups might be a lot of data to store by default.  Would you be open to a lower default like 3 or 5?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr-operator] gerlowskija commented on a change in pull request #359: Add ability to schedule recurring incremental backups

Posted by GitBox <gi...@apache.org>.
gerlowskija commented on a change in pull request #359:
URL: https://github.com/apache/solr-operator/pull/359#discussion_r740493719



##########
File path: api/v1beta1/solrbackup_types.go
##########
@@ -64,6 +71,29 @@ func (spec *SolrBackupSpec) withDefaults(backupName string) (changed bool) {
 	return changed
 }
 
+// BackupRecurrence defines the recurrence of the incremental backup
+type BackupRecurrence struct {
+	// Perform a backup on the given schedule, in CRON format.
+	//
+	// Multiple CRON syntaxes are supported
+	//   - Standard CRON (e.g. "CRON_TZ=Asia/Seoul 0 6 * * ?")
+	//   - Predefined Schedules (e.g. "@yearly", "@weekly", "@daily", etc.)
+	//   - Intervals (e.g. "@every 10h30m")
+	//
+	// For more information please check this reference:
+	// https://pkg.go.dev/github.com/robfig/cron/v3?utm_source=godoc#hdr-CRON_Expression_Format
+	Schedule string `json:"schedule"`
+
+	// Define the number of backup points to save for this backup at any given time.
+	// The oldest backups will be deleted if too many exist when a backup is taken.
+	// If not provided, this defaults to 10.
+	//
+	// +kubebuilder:default:=10
+	// +kubebuilder:validation:Minimum:=1
+	// +optional
+	MaxSaved int `json:"maxSaved,omitempty"`

Review comment:
       [0] This is mostly subjective preference, but I'd vote for a smaller default value here.  For collections with a lot of churn, 10 backups might be a lot of data to store by default.  Would you be open to a lower default like 3 or 5?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr-operator] HoustonPutman commented on a change in pull request #359: Add ability to schedule recurring incremental backups

Posted by GitBox <gi...@apache.org>.
HoustonPutman commented on a change in pull request #359:
URL: https://github.com/apache/solr-operator/pull/359#discussion_r741153892



##########
File path: api/v1beta1/solrbackup_types.go
##########
@@ -64,6 +71,29 @@ func (spec *SolrBackupSpec) withDefaults(backupName string) (changed bool) {
 	return changed
 }
 
+// BackupRecurrence defines the recurrence of the incremental backup
+type BackupRecurrence struct {
+	// Perform a backup on the given schedule, in CRON format.
+	//
+	// Multiple CRON syntaxes are supported
+	//   - Standard CRON (e.g. "CRON_TZ=Asia/Seoul 0 6 * * ?")
+	//   - Predefined Schedules (e.g. "@yearly", "@weekly", "@daily", etc.)
+	//   - Intervals (e.g. "@every 10h30m")
+	//
+	// For more information please check this reference:
+	// https://pkg.go.dev/github.com/robfig/cron/v3?utm_source=godoc#hdr-CRON_Expression_Format
+	Schedule string `json:"schedule"`
+
+	// Define the number of backup points to save for this backup at any given time.
+	// The oldest backups will be deleted if too many exist when a backup is taken.
+	// If not provided, this defaults to 10.
+	//
+	// +kubebuilder:default:=10
+	// +kubebuilder:validation:Minimum:=1
+	// +optional
+	MaxSaved int `json:"maxSaved,omitempty"`

Review comment:
       Very open. I literally just chose a number here. I am very happy to differ to whatever you think it should be. Give the call and it is done.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr-operator] HoustonPutman commented on a change in pull request #359: Add ability to schedule recurring incremental backups

Posted by GitBox <gi...@apache.org>.
HoustonPutman commented on a change in pull request #359:
URL: https://github.com/apache/solr-operator/pull/359#discussion_r741153892



##########
File path: api/v1beta1/solrbackup_types.go
##########
@@ -64,6 +71,29 @@ func (spec *SolrBackupSpec) withDefaults(backupName string) (changed bool) {
 	return changed
 }
 
+// BackupRecurrence defines the recurrence of the incremental backup
+type BackupRecurrence struct {
+	// Perform a backup on the given schedule, in CRON format.
+	//
+	// Multiple CRON syntaxes are supported
+	//   - Standard CRON (e.g. "CRON_TZ=Asia/Seoul 0 6 * * ?")
+	//   - Predefined Schedules (e.g. "@yearly", "@weekly", "@daily", etc.)
+	//   - Intervals (e.g. "@every 10h30m")
+	//
+	// For more information please check this reference:
+	// https://pkg.go.dev/github.com/robfig/cron/v3?utm_source=godoc#hdr-CRON_Expression_Format
+	Schedule string `json:"schedule"`
+
+	// Define the number of backup points to save for this backup at any given time.
+	// The oldest backups will be deleted if too many exist when a backup is taken.
+	// If not provided, this defaults to 10.
+	//
+	// +kubebuilder:default:=10
+	// +kubebuilder:validation:Minimum:=1
+	// +optional
+	MaxSaved int `json:"maxSaved,omitempty"`

Review comment:
       Very open. I literally just chose a number here. I am very happy to differ to whatever you think it should be. Give the call and it is done.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr-operator] HoustonPutman merged pull request #359: Add ability to schedule recurring incremental backups

Posted by GitBox <gi...@apache.org>.
HoustonPutman merged pull request #359:
URL: https://github.com/apache/solr-operator/pull/359


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr-operator] HoustonPutman commented on pull request #359: Add ability to schedule recurring incremental backups

Posted by GitBox <gi...@apache.org>.
HoustonPutman commented on pull request #359:
URL: https://github.com/apache/solr-operator/pull/359#issuecomment-963559221


   @gerlowskija, given that https://issues.apache.org/jira/browse/SOLR-15706 is not released yet, maybe we just wait to add in the `purge` call until the next Solr Operator release.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr-operator] gerlowskija commented on pull request #359: Add ability to schedule recurring incremental backups

Posted by GitBox <gi...@apache.org>.
gerlowskija commented on pull request #359:
URL: https://github.com/apache/solr-operator/pull/359#issuecomment-954904055


   Whoa man, this is awesome.  Thanks for picking this up - you took it right off my TODO list haha.
   
   I've just taken a glance so far but intend to give it a more thorough review this afternoon (or Monday at latest).
   
   One preliminary comment that I'm sure will put my foot solidly in my mouth once I actually review the code: I'm surprised to see that the scheduling functionality is exposed in the 'solrbackup' CRD as opposed to a new, separate resource type.
   
   Did you consider a separate CRD at all, and if so was there a reason you decided to do same-CRD?
   
   (When I thought of starting this work myself I assumed the backup-schedule functionality would have to be a separate CRD - so now I'm trying to recall whether that was just an assumption on my part or whether there was a particular reason behind that thought haha
   
   Maybe it was jut a conceptual misunderstanding or inflexibility on my part?  In my head I've always conceptualized 'solrbackup' as being scoped to tracking the progress of a single, one-off backup.  A user might be surprised that some solrbackup resources represent one-offs that will never impact the cluster again, and others represent ongoing periodic work that'll continue indefinitely.)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr-operator] HoustonPutman commented on pull request #359: Add ability to schedule recurring incremental backups

Posted by GitBox <gi...@apache.org>.
HoustonPutman commented on pull request #359:
URL: https://github.com/apache/solr-operator/pull/359#issuecomment-957772393


   1. I'm content with a `purgeEvery` option in Solr. I'm not sure I understand why you would want un-garbage collected index files laying about, but that's definitely a different issue. We can definitely go ahead and purge after every backup until the `purgeEvery` option is added. (Oh now I realize you mean a `purgeEvery` option in the CRD. I think we can assume that would be equivalent to `saveEvery`.)
   I did talk with @anshumg about a ticket that saw some issues with purging that does make me a bit wary to do it always. https://issues.apache.org/jira/browse/SOLR-15706
   We should make sure that's not an issue before going forward with that.
   2. That's a good idea. I do think I want to have it "enabled" by default. Which is easy to do with those kubebuilder annotations.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr-operator] HoustonPutman commented on pull request #359: Add ability to schedule recurring incremental backups

Posted by GitBox <gi...@apache.org>.
HoustonPutman commented on pull request #359:
URL: https://github.com/apache/solr-operator/pull/359#issuecomment-957835892


   > When you mention "saveEvery" I assume you mean maxSaved? Or is there some other "saveEvery" param I missed?
   
   I mis-typed. I meant `maxSaved` 🙂 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org