You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@uniffle.apache.org by GitBox <gi...@apache.org> on 2022/08/22 07:44:10 UTC
[GitHub] [incubator-uniffle] wangao1236 opened a new pull request, #175: [ISSUE-48][FEATURE][FOLLOW UP] Add RemoteShuffleService CRD
wangao1236 opened a new pull request, #175:
URL: https://github.com/apache/incubator-uniffle/pull/175
<!--
Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://github.com/Tencent/Firestorm/blob/master/CONTRIBUTING.md
2. Ensure you have added or run the appropriate tests for your PR
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP]XXXX Your PR title ...'.
4. Be sure to keep the PR description updated to reflect all changes.
5. Please write your PR title to summarize what this PR proposes.
6. If possible, provide a concise example to reproduce the issue for a faster review.
-->
### What changes were proposed in this pull request?
for issue #48
I add a crd object called RemoteShuffleService this time, and I will submit the controller of the crd in next pr.
### Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, you can clarify why it is a bug.
-->
### Does this PR introduce _any_ user-facing change?
<!--
Note that it means *any* user-facing change including all aspects such as the documentation fix.
If yes, please clarify the previous behavior and the change this PR proposes - provide the console output, description and/or an example to show the behavior difference if possible.
If possible, please also clarify if this is a user-facing change compared to the released versions or within the unreleased branches such as master.
If no, write 'No'.
-->
### How was this patch tested?
<!--
If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible.
If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
If tests were not added, please describe why they were not added and/or why it was difficult to add.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] wangao1236 commented on a diff in pull request #175: [ISSUE-48][FEATURE][FOLLOW UP] Add RemoteShuffleService CRD
Posted by GitBox <gi...@apache.org>.
wangao1236 commented on code in PR #175:
URL: https://github.com/apache/incubator-uniffle/pull/175#discussion_r954745773
##########
deploy/kubernetes/operator/api/uniffle/v1alpha1/remoteshuffleservice_types.go:
##########
@@ -18,49 +18,303 @@
package v1alpha1
import (
+ "k8s.io/api/autoscaling/v2beta2"
+ corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
+type ShuffleServerUpgradeStrategyType string
+
+const (
+ PartitionUpgrade ShuffleServerUpgradeStrategyType = "PartitionUpgrade"
+ SpecificUpgrade ShuffleServerUpgradeStrategyType = "SpecificUpgrade"
+ FullUpgrade ShuffleServerUpgradeStrategyType = "FullUpgrade"
+ FullRestart ShuffleServerUpgradeStrategyType = "FullRestart"
+)
+
+type RSSPhase string
+
+const (
+ // RSSPending represents RSS object is pending.
+ RSSPending RSSPhase = "Pending"
+ // RSSRunning represents RSS object is running normally.
+ RSSRunning RSSPhase = "Running"
+ // RSSTerminating represents RSS object is terminating.
+ RSSTerminating RSSPhase = "Terminating"
+ // RSSFailed represents RSS object has been failed.
+ RSSFailed RSSPhase = "Failed"
+ // RSSUpgrading represents RSS object is upgrading.
+ RSSUpgrading RSSPhase = "Upgrading"
+)
+
// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN!
// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized.
-// RemoteshuffleserviceSpec defines the desired state of Remoteshuffleservice
-type RemoteshuffleserviceSpec struct {
- // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster
- // Important: Run "make" to regenerate code after modifying this file
+// RemoteShuffleServiceSpec defines the desired state of RemoteShuffleService.
+type RemoteShuffleServiceSpec struct {
+ // Coordinator contains configurations of the coordinators.
+ Coordinator *CoordinatorConfig `json:"coordinator"`
- // Foo is an example field of Remoteshuffleservice. Edit remoteshuffleservice_types.go to remove/update
- Foo string `json:"foo,omitempty"`
+ // ShuffleServer contains configuration of the shuffle servers.
+ ShuffleServer *ShuffleServerConfig `json:"shuffleServer"`
+
+ // ConfigMapName indicates configMap name stores configurations of coordinators and shuffle servers.
+ ConfigMapName string `json:"configMapName"`
}
-// RemoteshuffleserviceStatus defines the observed state of Remoteshuffleservice
-type RemoteshuffleserviceStatus struct {
+// CoordinatorConfig records configuration used to generate workload of coordination.
+type CoordinatorConfig struct {
+ *CommonConfig `json:",inline"`
+
+ // +kubebuilder:default:=true
+ // Sync indicates whether we need to sync configurations to the running coordinators.
+ // +optional
+ Sync *bool `json:"sync,omitempty"`
+
+ // +kubebuilder:default:=2
+ // Count is the number of coordinator workloads to be generated.
+ // +optional
+ Count *int32 `json:"count,omitempty"`
+
+ // +kubebuilder:default:=1
+ // Replicas is the initial replicas of coordinators.
+ // +optional
+ Replicas *int32 `json:"replicas,omitempty"`
+
+ // +kubebuilder:default:=19997
+ // RPCPort defines rpc port used by coordinators.
+ // +optional
+ RPCPort *int32 `json:"rpcPort,omitempty"`
+
+ // +kubebuilder:default:=19996
+ // HTTPPort defines http port used by coordinators.
+ // +optional
+ HTTPPort *int32 `json:"httpPort,omitempty"`
+
+ // +kubebuilder:default:=/config/exclude_nodes
+ // ExcludeNodesFilePath indicates exclude nodes file path in coordinators' containers.
+ // +optional
+ ExcludeNodesFilePath string `json:"excludeNodesFilePath,omitempty"`
+
+ // RPCNodePort defines rpc port of node port service used for coordinators' external access.
+ RPCNodePort []int32 `json:"rpcNodePort"`
+
+ // HTTPNodePort defines http port of node port service used for coordinators' external access.
+ HTTPNodePort []int32 `json:"httpNodePort"`
+}
+
+// ShuffleServerConfig records configuration used to generate workload of shuffle servers.
+type ShuffleServerConfig struct {
+ *CommonConfig `json:",inline"`
+
+ // +kubebuilder:default:=false
+ // Sync indicates whether we need to sync configurations to the running shuffle servers.
+ // +optional
+ Sync *bool `json:"sync,omitempty"`
+
+ // +kubebuilder:default:=1
+ // Replicas is the initial replicas of shuffle servers.
+ // +optional
+ Replicas *int32 `json:"replicas,omitempty"`
+
+ // +kubebuilder:default:=19997
+ // RPCPort defines rpc port used by shuffle servers.
+ // +optional
+ RPCPort *int32 `json:"rpcPort,omitempty"`
+
+ // +kubebuilder:default:=19996
+ // HTTPPort defines http port used by shuffle servers.
+ // +optional
+ HTTPPort *int32 `json:"httpPort,omitempty"`
+
+ // RPCNodePort defines rpc port of node port service used for shuffle servers' external access.
+ // +optional
+ RPCNodePort *int32 `json:"rpcNodePort,omitempty"`
+
+ // HTTPNodePort defines http port of node port service used for shuffle servers' external access.
+ // +optional
+ HTTPNodePort *int32 `json:"httpNodePort,omitempty"`
+
+ // UpgradeStrategy defines upgrade strategy of shuffle servers.
+ UpgradeStrategy *ShuffleServerUpgradeStrategy `json:"upgradeStrategy"`
+}
+
+// ShuffleServerUpgradeStrategy defines upgrade strategy of shuffle servers.
+type ShuffleServerUpgradeStrategy struct {
+ // Type represents upgrade type of shuffle servers, including partition, specific copy and full upgrade.
+ Type ShuffleServerUpgradeStrategyType `json:"type"`
+
+ // Partition means the minimum number that needs to be upgraded, the copies whose numbers are greater than or
+ // equal to this number needs to be upgraded.
+ // +optional
+ Partition *int32 `json:"partition,omitempty"`
+
+ // SpecificNames indicates the specific pod names of shuffle servers which we want to upgrade.
+ // +optional
+ SpecificNames []string `json:"specificNames,omitempty"`
+}
+
+// RSSAutoscaler describes the desired functionality of the HPA object to be generated,
+// which automatically manages the replica count of any resource implementing the scale
+// subresource based on the metrics specified.
+type RSSAutoscaler struct {
Review Comment:
Ok, after discussion, I will remove this field first and add it in the future according to the requirements.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] jerqi commented on a diff in pull request #175: [ISSUE-48][FEATURE][FOLLOW UP] Add RemoteShuffleService CRD
Posted by GitBox <gi...@apache.org>.
jerqi commented on code in PR #175:
URL: https://github.com/apache/incubator-uniffle/pull/175#discussion_r952232298
##########
deploy/kubernetes/operator/pkg/.gitkeep:
##########
@@ -0,0 +1,16 @@
+#
Review Comment:
If operator directory have files, could we remove this file?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] codecov-commenter commented on pull request #175: [ISSUE-48][FEATURE][FOLLOW UP] Add RemoteShuffleService CRD
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #175:
URL: https://github.com/apache/incubator-uniffle/pull/175#issuecomment-1221996643
# [Codecov](https://codecov.io/gh/apache/incubator-uniffle/pull/175?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#175](https://codecov.io/gh/apache/incubator-uniffle/pull/175?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (aa897a9) into [master](https://codecov.io/gh/apache/incubator-uniffle/commit/74cd40b9fc5f41b8b04fbaac9bab31985e7b6d15?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (74cd40b) will **decrease** coverage by `1.35%`.
> The diff coverage is `n/a`.
```diff
@@ Coverage Diff @@
## master #175 +/- ##
============================================
- Coverage 58.29% 56.93% -1.36%
+ Complexity 1262 1184 -78
============================================
Files 158 149 -9
Lines 8397 7902 -495
Branches 779 749 -30
============================================
- Hits 4895 4499 -396
+ Misses 3251 3161 -90
+ Partials 251 242 -9
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-uniffle/pull/175?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...org/apache/uniffle/server/ShuffleFlushManager.java](https://codecov.io/gh/apache/incubator-uniffle/pull/175/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS91bmlmZmxlL3NlcnZlci9TaHVmZmxlRmx1c2hNYW5hZ2VyLmphdmE=) | `77.65% <0.00%> (-1.68%)` | :arrow_down: |
| [.../java/org/apache/spark/shuffle/RssSparkConfig.java](https://codecov.io/gh/apache/incubator-uniffle/pull/175/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y2xpZW50LXNwYXJrL2NvbW1vbi9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc3Bhcmsvc2h1ZmZsZS9Sc3NTcGFya0NvbmZpZy5qYXZh) | | |
| [...k/shuffle/writer/WrappedByteArrayOutputStream.java](https://codecov.io/gh/apache/incubator-uniffle/pull/175/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y2xpZW50LXNwYXJrL2NvbW1vbi9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc3Bhcmsvc2h1ZmZsZS93cml0ZXIvV3JhcHBlZEJ5dGVBcnJheU91dHB1dFN0cmVhbS5qYXZh) | | |
| [...e/spark/shuffle/reader/RssShuffleDataIterator.java](https://codecov.io/gh/apache/incubator-uniffle/pull/175/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y2xpZW50LXNwYXJrL2NvbW1vbi9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc3Bhcmsvc2h1ZmZsZS9yZWFkZXIvUnNzU2h1ZmZsZURhdGFJdGVyYXRvci5qYXZh) | | |
| [.../org/apache/spark/shuffle/writer/WriterBuffer.java](https://codecov.io/gh/apache/incubator-uniffle/pull/175/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y2xpZW50LXNwYXJrL2NvbW1vbi9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc3Bhcmsvc2h1ZmZsZS93cml0ZXIvV3JpdGVyQnVmZmVyLmphdmE=) | | |
| [...pache/spark/shuffle/writer/WriteBufferManager.java](https://codecov.io/gh/apache/incubator-uniffle/pull/175/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y2xpZW50LXNwYXJrL2NvbW1vbi9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc3Bhcmsvc2h1ZmZsZS93cml0ZXIvV3JpdGVCdWZmZXJNYW5hZ2VyLmphdmE=) | | |
| [...che/spark/shuffle/writer/BufferManagerOptions.java](https://codecov.io/gh/apache/incubator-uniffle/pull/175/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y2xpZW50LXNwYXJrL2NvbW1vbi9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc3Bhcmsvc2h1ZmZsZS93cml0ZXIvQnVmZmVyTWFuYWdlck9wdGlvbnMuamF2YQ==) | | |
| [...org/apache/spark/shuffle/RssSparkShuffleUtils.java](https://codecov.io/gh/apache/incubator-uniffle/pull/175/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y2xpZW50LXNwYXJrL2NvbW1vbi9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc3Bhcmsvc2h1ZmZsZS9Sc3NTcGFya1NodWZmbGVVdGlscy5qYXZh) | | |
| [...ava/org/apache/spark/shuffle/RssShuffleHandle.java](https://codecov.io/gh/apache/incubator-uniffle/pull/175/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y2xpZW50LXNwYXJrL2NvbW1vbi9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc3Bhcmsvc2h1ZmZsZS9Sc3NTaHVmZmxlSGFuZGxlLmphdmE=) | | |
| [...org/apache/spark/shuffle/writer/AddBlockEvent.java](https://codecov.io/gh/apache/incubator-uniffle/pull/175/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y2xpZW50LXNwYXJrL2NvbW1vbi9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc3Bhcmsvc2h1ZmZsZS93cml0ZXIvQWRkQmxvY2tFdmVudC5qYXZh) | | |
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] thousandhu commented on pull request #175: [ISSUE-48][FEATURE][FOLLOW UP] Add RemoteShuffleService CRD
Posted by GitBox <gi...@apache.org>.
thousandhu commented on PR #175:
URL: https://github.com/apache/incubator-uniffle/pull/175#issuecomment-1227147101
LGTM , thanks @wangao1236
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] jerqi commented on pull request #175: [ISSUE-48][FEATURE][FOLLOW UP] Add RemoteShuffleService CRD
Posted by GitBox <gi...@apache.org>.
jerqi commented on PR #175:
URL: https://github.com/apache/incubator-uniffle/pull/175#issuecomment-1221986374
@zuston @thousandhu @czy006 Could you help me review this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] wangao1236 commented on a diff in pull request #175: [ISSUE-48][FEATURE][FOLLOW UP] Add RemoteShuffleService CRD
Posted by GitBox <gi...@apache.org>.
wangao1236 commented on code in PR #175:
URL: https://github.com/apache/incubator-uniffle/pull/175#discussion_r952233176
##########
deploy/kubernetes/operator/pkg/.gitkeep:
##########
@@ -0,0 +1,16 @@
+#
Review Comment:
Yes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] thousandhu commented on a diff in pull request #175: [ISSUE-48][FEATURE][FOLLOW UP] Add RemoteShuffleService CRD
Posted by GitBox <gi...@apache.org>.
thousandhu commented on code in PR #175:
URL: https://github.com/apache/incubator-uniffle/pull/175#discussion_r953525122
##########
deploy/kubernetes/operator/api/uniffle/v1alpha1/remoteshuffleservice_types.go:
##########
@@ -18,49 +18,303 @@
package v1alpha1
import (
+ "k8s.io/api/autoscaling/v2beta2"
+ corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
+type ShuffleServerUpgradeStrategyType string
+
+const (
+ PartitionUpgrade ShuffleServerUpgradeStrategyType = "PartitionUpgrade"
+ SpecificUpgrade ShuffleServerUpgradeStrategyType = "SpecificUpgrade"
+ FullUpgrade ShuffleServerUpgradeStrategyType = "FullUpgrade"
+ FullRestart ShuffleServerUpgradeStrategyType = "FullRestart"
+)
+
+type RSSPhase string
+
+const (
+ // RSSPending represents RSS object is pending.
+ RSSPending RSSPhase = "Pending"
+ // RSSRunning represents RSS object is running normally.
+ RSSRunning RSSPhase = "Running"
+ // RSSTerminating represents RSS object is terminating.
+ RSSTerminating RSSPhase = "Terminating"
+ // RSSFailed represents RSS object has been failed.
+ RSSFailed RSSPhase = "Failed"
+ // RSSUpgrading represents RSS object is upgrading.
+ RSSUpgrading RSSPhase = "Upgrading"
+)
+
// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN!
// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized.
-// RemoteshuffleserviceSpec defines the desired state of Remoteshuffleservice
-type RemoteshuffleserviceSpec struct {
- // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster
- // Important: Run "make" to regenerate code after modifying this file
+// RemoteShuffleServiceSpec defines the desired state of RemoteShuffleService.
+type RemoteShuffleServiceSpec struct {
+ // Coordinator contains configurations of the coordinators.
+ Coordinator *CoordinatorConfig `json:"coordinator"`
- // Foo is an example field of Remoteshuffleservice. Edit remoteshuffleservice_types.go to remove/update
- Foo string `json:"foo,omitempty"`
+ // ShuffleServer contains configuration of the shuffle servers.
+ ShuffleServer *ShuffleServerConfig `json:"shuffleServer"`
+
+ // ConfigMapName indicates configMap name stores configurations of coordinators and shuffle servers.
+ ConfigMapName string `json:"configMapName"`
}
-// RemoteshuffleserviceStatus defines the observed state of Remoteshuffleservice
-type RemoteshuffleserviceStatus struct {
+// CoordinatorConfig records configuration used to generate workload of coordination.
+type CoordinatorConfig struct {
+ *CommonConfig `json:",inline"`
+
+ // +kubebuilder:default:=true
+ // Sync indicates whether we need to sync configurations to the running coordinators.
+ // +optional
+ Sync *bool `json:"sync,omitempty"`
+
+ // +kubebuilder:default:=2
+ // Count is the number of coordinator workloads to be generated.
+ // +optional
+ Count *int32 `json:"count,omitempty"`
+
+ // +kubebuilder:default:=1
+ // Replicas is the initial replicas of coordinators.
+ // +optional
+ Replicas *int32 `json:"replicas,omitempty"`
+
+ // +kubebuilder:default:=19997
+ // RPCPort defines rpc port used by coordinators.
+ // +optional
+ RPCPort *int32 `json:"rpcPort,omitempty"`
+
+ // +kubebuilder:default:=19996
+ // HTTPPort defines http port used by coordinators.
+ // +optional
+ HTTPPort *int32 `json:"httpPort,omitempty"`
+
+ // +kubebuilder:default:=/config/exclude_nodes
+ // ExcludeNodesFilePath indicates exclude nodes file path in coordinators' containers.
+ // +optional
+ ExcludeNodesFilePath string `json:"excludeNodesFilePath,omitempty"`
+
+ // RPCNodePort defines rpc port of node port service used for coordinators' external access.
+ RPCNodePort []int32 `json:"rpcNodePort"`
+
+ // HTTPNodePort defines http port of node port service used for coordinators' external access.
+ HTTPNodePort []int32 `json:"httpNodePort"`
+}
+
+// ShuffleServerConfig records configuration used to generate workload of shuffle servers.
+type ShuffleServerConfig struct {
+ *CommonConfig `json:",inline"`
+
+ // +kubebuilder:default:=false
+ // Sync indicates whether we need to sync configurations to the running shuffle servers.
+ // +optional
+ Sync *bool `json:"sync,omitempty"`
+
+ // +kubebuilder:default:=1
+ // Replicas is the initial replicas of shuffle servers.
+ // +optional
+ Replicas *int32 `json:"replicas,omitempty"`
+
+ // +kubebuilder:default:=19997
+ // RPCPort defines rpc port used by shuffle servers.
+ // +optional
+ RPCPort *int32 `json:"rpcPort,omitempty"`
+
+ // +kubebuilder:default:=19996
+ // HTTPPort defines http port used by shuffle servers.
+ // +optional
+ HTTPPort *int32 `json:"httpPort,omitempty"`
+
+ // RPCNodePort defines rpc port of node port service used for shuffle servers' external access.
+ // +optional
+ RPCNodePort *int32 `json:"rpcNodePort,omitempty"`
+
+ // HTTPNodePort defines http port of node port service used for shuffle servers' external access.
+ // +optional
+ HTTPNodePort *int32 `json:"httpNodePort,omitempty"`
+
+ // UpgradeStrategy defines upgrade strategy of shuffle servers.
+ UpgradeStrategy *ShuffleServerUpgradeStrategy `json:"upgradeStrategy"`
+}
+
+// ShuffleServerUpgradeStrategy defines upgrade strategy of shuffle servers.
+type ShuffleServerUpgradeStrategy struct {
+ // Type represents upgrade type of shuffle servers, including partition, specific copy and full upgrade.
+ Type ShuffleServerUpgradeStrategyType `json:"type"`
+
+ // Partition means the minimum number that needs to be upgraded, the copies whose numbers are greater than or
+ // equal to this number needs to be upgraded.
+ // +optional
+ Partition *int32 `json:"partition,omitempty"`
+
+ // SpecificNames indicates the specific pod names of shuffle servers which we want to upgrade.
+ // +optional
+ SpecificNames []string `json:"specificNames,omitempty"`
+}
+
+// RSSAutoscaler describes the desired functionality of the HPA object to be generated,
+// which automatically manages the replica count of any resource implementing the scale
+// subresource based on the metrics specified.
+type RSSAutoscaler struct {
Review Comment:
I didn't see hpa in our design doc. Should we discuss it first before adding it into crd?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] jerqi merged pull request #175: [ISSUE-48][FEATURE][FOLLOW UP] Add RemoteShuffleService CRD
Posted by GitBox <gi...@apache.org>.
jerqi merged PR #175:
URL: https://github.com/apache/incubator-uniffle/pull/175
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org