You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kie.apache.org by "jordigilh (via GitHub)" <gi...@apache.org> on 2023/11/28 18:43:12 UTC
[PR] Kogito 9940 add job service sonataflowplatform [incubator-kie-kogito-serverless-operator]
jordigilh opened a new pull request, #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311
* Adds a new field to the `ServiceSpec` in the SonataflowPlatform CRD for a Job Service, reusing the same type as the `DataIndex` field.
* Implements the reconciliation logic for the Job Service.
* Adds 2 new unit tests specific for the Job Service and another one for the use case where both DataIndex and Job Service fields are populated with an empty struct.
@tchughesiv @ricardozanini PTAL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1409755648
##########
api/v1alpha08/sonataflowplatform_services_types.go:
##########
@@ -19,6 +19,9 @@ type ServicesPlatformSpec struct {
// Deploys the Data Index service for use by "prod" profile workflows.
// +optional
DataIndex *ServiceSpec `json:"dataIndex,omitempty"`
+ // Deploys the Job service for use by "prod" profile workflows.
+ // +optional
+ JobService *ServiceSpec `json:"job,omitempty"`
Review Comment:
good catch! I'll rename it as agreed to `jobService`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "masayag (via GitHub)" <gi...@apache.org>.
masayag commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1416046675
##########
controllers/platform/services.go:
##########
@@ -58,30 +55,32 @@ func (action *serviceAction) Handle(ctx context.Context, platform *operatorapi.S
return nil, err
}
- if err := createDataIndexComponents(ctx, action.client, platform); err != nil {
- return nil, err
+ if platform.Spec.Services.DataIndex != nil {
Review Comment:
Thank you for the reference.
In which case should this be set to false, assuming the workflow's profile is set to `prod`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "tchughesiv (via GitHub)" <gi...@apache.org>.
tchughesiv commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1422748685
##########
controllers/platform/services.go:
##########
@@ -58,30 +55,32 @@ func (action *serviceAction) Handle(ctx context.Context, platform *operatorapi.S
return nil, err
}
- if err := createDataIndexComponents(ctx, action.client, platform); err != nil {
- return nil, err
+ if platform.Spec.Services.DataIndex != nil {
Review Comment:
I also think this all sounds reasonable, with `RollingUpdate` as the default when this new "flag" is not set. @ricardozanini I don't think the cluster-scoped changes would need to wait for this though... what is your thinking with that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "wmedvede (via GitHub)" <gi...@apache.org>.
wmedvede commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1424323276
##########
controllers/platform/services/services.go:
##########
@@ -0,0 +1,298 @@
+// Copyright 2023 Red Hat, Inc. and/or its affiliates
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package services
+
+import (
+ "fmt"
+ "strconv"
+
+ corev1 "k8s.io/api/core/v1"
+ "k8s.io/apimachinery/pkg/api/resource"
+
+ operatorapi "github.com/apache/incubator-kie-kogito-serverless-operator/api/v1alpha08"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles/common/constants"
+
+ "github.com/apache/incubator-kie-kogito-serverless-operator/version"
+ "github.com/imdario/mergo"
+)
+
+type Platform interface {
+ // GetContainerName returns the name of the service's container in the deployment.
+ GetContainerName() string
+ // GetServiceImageName returns the image name of the service's container. It takes in the service and persistence types and returns a string
+ // that contains the FQDN of the image, including the tag.
+ GetServiceImageName(persistenceName string) string
+ // GetServiceName returns the name of the kubernetes service prefixed with the platform name
+ GetServiceName() string
+ // GetServiceCmName returns the name of the configmap associated to the service
+ GetServiceCmName() string
+ // GetEnvironmentVariables returns the env variables to be injected to the service container
+ GetEnvironmentVariables() []corev1.EnvVar
+ // GetResourceLimits returns the pod's memory and CPU resource requirements
+ // Values for job service taken from
+ // https://github.com/parodos-dev/orchestrator-helm-chart/blob/52d09eda56fdbed3060782df29847c97f172600f/charts/orchestrator/values.yaml#L68-L72
+ GetPodResourceRequirements() corev1.ResourceRequirements
+ // GetReplicaCountForService Returns the default pod replica count for the given service
+ GetReplicaCount() int32
Review Comment:
@jordigilh At the time of processing this value, for the case of the jobs-service, we must take into account that in its current start of the art, it's a singleton service. So we only create 1 replica, no matter if the user indicated another value, we can just overwrite and use 1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1424843077
##########
controllers/platform/services/properties.go:
##########
@@ -0,0 +1,98 @@
+// Copyright 2023 Apache Software Foundation (ASF)
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package services
+
+import (
+ "fmt"
+ "strings"
+
+ operatorapi "github.com/apache/incubator-kie-kogito-serverless-operator/api/v1alpha08"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles/common/constants"
+
+ "github.com/magiconair/properties"
+)
+
+func generateReactiveURL(postgresSpec *operatorapi.PersistencePostgreSql, schema string, namespace string, dbName string, port int) string {
+ if len(postgresSpec.JdbcUrl) > 0 && strings.Contains(postgresSpec.JdbcUrl, "currentSchema=") {
+ return strings.Replace(strings.TrimPrefix(postgresSpec.JdbcUrl, "jdbc:"), "currentSchema=", "search_path=", -1)
Review Comment:
Updated code to parse the URL and generate it based on the contents of the parsed object.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "wmedvede (via GitHub)" <gi...@apache.org>.
wmedvede commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1425116853
##########
controllers/platform/services/properties.go:
##########
@@ -0,0 +1,131 @@
+// Copyright 2023 Apache Software Foundation (ASF)
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package services
+
+import (
+ "fmt"
+ "net/url"
+ "strings"
+
+ operatorapi "github.com/apache/incubator-kie-kogito-serverless-operator/api/v1alpha08"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles/common/constants"
+
+ "github.com/magiconair/properties"
+)
+
+func generateReactiveURL(postgresSpec *operatorapi.PersistencePostgreSql, schema string, namespace string, dbName string, port int) (string, error) {
+ if len(postgresSpec.JdbcUrl) > 0 {
+ s := strings.TrimLeft(postgresSpec.JdbcUrl, "jdbc:")
+ u, err := url.Parse(s)
+ if err != nil {
+ return "", err
+ }
+ ret := fmt.Sprintf("%s://", u.Scheme)
+ if len(u.User.Username()) > 0 {
+ p, ok := u.User.Password()
+ if ok {
+ ret = fmt.Sprintf("%s%s:%s@", ret, u.User.Username(), p)
+ }
+ }
+ ret = fmt.Sprintf("%s%s%s?", ret, u.Host, u.Path)
+ kv, err := url.ParseQuery(u.RawQuery)
+ if err != nil {
+ return "", err
+ }
+ spv := schema
Review Comment:
I believe that in this case we are analyzing the JdbcUrl that was provided by the user, and thus, we should infer the schema from it, but not default to svp := schema, not good in my opinion.
why?
if the JdbcUrl has no schema, flyway will create the tables in the database default schema, since the tables are created by using the JdbcUrl non reactive datasource. And, then, the reactive datasource will point to a different schema.
See that it remains consistent to what we have in:
configurePostgreSqlEnv
if len(postgresql.JdbcUrl) > 0 {
dataSourceUrl = postgresql.JdbcUrl //here we take the jdbc url as is.
}
##########
controllers/platform/services/properties.go:
##########
@@ -0,0 +1,131 @@
+// Copyright 2023 Apache Software Foundation (ASF)
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package services
+
+import (
+ "fmt"
+ "net/url"
+ "strings"
+
+ operatorapi "github.com/apache/incubator-kie-kogito-serverless-operator/api/v1alpha08"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles/common/constants"
+
+ "github.com/magiconair/properties"
+)
+
+func generateReactiveURL(postgresSpec *operatorapi.PersistencePostgreSql, schema string, namespace string, dbName string, port int) (string, error) {
+ if len(postgresSpec.JdbcUrl) > 0 {
+ s := strings.TrimLeft(postgresSpec.JdbcUrl, "jdbc:")
+ u, err := url.Parse(s)
+ if err != nil {
+ return "", err
+ }
+ ret := fmt.Sprintf("%s://", u.Scheme)
+ if len(u.User.Username()) > 0 {
+ p, ok := u.User.Password()
+ if ok {
+ ret = fmt.Sprintf("%s%s:%s@", ret, u.User.Username(), p)
+ }
+ }
+ ret = fmt.Sprintf("%s%s%s?", ret, u.Host, u.Path)
+ kv, err := url.ParseQuery(u.RawQuery)
+ if err != nil {
+ return "", err
+ }
+ spv := schema
+ if v, ok := kv["search_path"]; ok {
+ for _, val := range v {
+ if len(val) != 0 {
+ spv = v[0]
+ }
+ }
+ } else if v, ok := kv["currentSchema"]; ok {
+ for _, val := range v {
+ if len(val) != 0 {
+ spv = v[0]
Review Comment:
here we try to infer the schema, good!
##########
controllers/platform/services/properties.go:
##########
@@ -0,0 +1,131 @@
+// Copyright 2023 Apache Software Foundation (ASF)
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package services
+
+import (
+ "fmt"
+ "net/url"
+ "strings"
+
+ operatorapi "github.com/apache/incubator-kie-kogito-serverless-operator/api/v1alpha08"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles/common/constants"
+
+ "github.com/magiconair/properties"
+)
+
+func generateReactiveURL(postgresSpec *operatorapi.PersistencePostgreSql, schema string, namespace string, dbName string, port int) (string, error) {
+ if len(postgresSpec.JdbcUrl) > 0 {
+ s := strings.TrimLeft(postgresSpec.JdbcUrl, "jdbc:")
+ u, err := url.Parse(s)
+ if err != nil {
+ return "", err
+ }
+ ret := fmt.Sprintf("%s://", u.Scheme)
+ if len(u.User.Username()) > 0 {
+ p, ok := u.User.Password()
+ if ok {
+ ret = fmt.Sprintf("%s%s:%s@", ret, u.User.Username(), p)
+ }
+ }
+ ret = fmt.Sprintf("%s%s%s?", ret, u.Host, u.Path)
+ kv, err := url.ParseQuery(u.RawQuery)
+ if err != nil {
+ return "", err
+ }
+ spv := schema
+ if v, ok := kv["search_path"]; ok {
+ for _, val := range v {
+ if len(val) != 0 {
+ spv = v[0]
Review Comment:
here we try to infer the schema, good!
##########
controllers/platform/services/properties.go:
##########
@@ -0,0 +1,131 @@
+// Copyright 2023 Apache Software Foundation (ASF)
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package services
+
+import (
+ "fmt"
+ "net/url"
+ "strings"
+
+ operatorapi "github.com/apache/incubator-kie-kogito-serverless-operator/api/v1alpha08"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles/common/constants"
+
+ "github.com/magiconair/properties"
+)
+
+func generateReactiveURL(postgresSpec *operatorapi.PersistencePostgreSql, schema string, namespace string, dbName string, port int) (string, error) {
+ if len(postgresSpec.JdbcUrl) > 0 {
+ s := strings.TrimLeft(postgresSpec.JdbcUrl, "jdbc:")
+ u, err := url.Parse(s)
+ if err != nil {
+ return "", err
+ }
+ ret := fmt.Sprintf("%s://", u.Scheme)
+ if len(u.User.Username()) > 0 {
+ p, ok := u.User.Password()
+ if ok {
+ ret = fmt.Sprintf("%s%s:%s@", ret, u.User.Username(), p)
+ }
+ }
+ ret = fmt.Sprintf("%s%s%s?", ret, u.Host, u.Path)
+ kv, err := url.ParseQuery(u.RawQuery)
+ if err != nil {
+ return "", err
+ }
+ spv := schema
+ if v, ok := kv["search_path"]; ok {
+ for _, val := range v {
+ if len(val) != 0 {
+ spv = v[0]
+ }
+ }
+ } else if v, ok := kv["currentSchema"]; ok {
+ for _, val := range v {
+ if len(val) != 0 {
+ spv = v[0]
+ }
+ }
+ }
+ return fmt.Sprintf("%ssearch_path=%s", ret, spv), nil
Review Comment:
when we reach this point, if we couldn't infer the schema from the JdbcUrl, we don't set it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "wmedvede (via GitHub)" <gi...@apache.org>.
wmedvede commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1860503486
> @wmedvede @jordigilh `mp.messaging.outgoing.kogito-processinstances-events.url` should only be configured within the workflow properties when DI is enabled in the platform. with this PR it now appears it's being added to the DI/JS service properties as well. this is the result of the property refactoring that's been done.
>
> `mp.messaging.outgoing.kogito-processinstances-events.url` && `mp.messaging.outgoing.kogito-job-service-job-request-events.url` are **client-side** configs and should only be set in the workflow properties when the corresponding service is enabled in the platform.
yep, I think as soon this prop is not added any more we'll get both services starting.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "wmedvede (via GitHub)" <gi...@apache.org>.
wmedvede commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1858245371
Hi @jordigilh , I'll try to answer to your question above regarding the properties here:
**Jobs Service specific properties:**
kogito.service.url (cluster dns named url to self e.g. http://jobs-service-postgresql)
quarkus.datasource.jdbc.url
quarkus.datasource.reactive.url
health.check."org.kie.kogito.jobs.service.messaging.http.health.knative.KSinkInjectionHealthCheck".enabled=false
**Then, if data-index is present in the platform, we also add**
kogito.jobs-service.http.job-status-change-events=true
#cluster dns of the data index, which operator should know. example below
mp.messaging.outgoing.kogito-job-service-job-status-events-http.url=http://data-index-service-postgresql/jobs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "tchughesiv (via GitHub)" <gi...@apache.org>.
tchughesiv commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1856552150
> By changing the `ServiceRef` to be a pointer, and adjusting the code to take that into account, solves the issue without having to change the limits:
@jordigilh that makes sense... i think i only tested this in the OpenShift UI so this didn't bite me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "tchughesiv (via GitHub)" <gi...@apache.org>.
tchughesiv commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1411086130
##########
controllers/platform/services.go:
##########
@@ -58,30 +60,72 @@ func (action *serviceAction) Handle(ctx context.Context, platform *operatorapi.S
return nil, err
}
- if err := createDataIndexComponents(ctx, action.client, platform); err != nil {
- return nil, err
+ if platform.Spec.Services.DataIndex != nil {
Review Comment:
agreed we should enhance the SonataFlowPlatform's status to account for service deployments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "ricardozanini (via GitHub)" <gi...@apache.org>.
ricardozanini commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1830650127
Can you run:
```
make generate-all
make vet fmt
```
Should pass the CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1408267628
##########
controllers/profiles/common/app_properties.go:
##########
@@ -45,27 +45,59 @@ const (
kafkaSmallRyeHealthProperty = "quarkus.smallrye-health.check.\"io.quarkus.kafka.client.health.KafkaHealthCheck\".enabled"
dataIndexServiceUrlProtocol = "http"
- DataIndexImageBase = "quay.io/kiegroup/kogito-data-index-"
- DataIndexName = "data-index-service"
-
- PersistenceTypeEphemeral = "ephemeral"
- PersistenceTypePostgressql = "postgresql"
-
+ imageNamePrefix = "quay.io/kiegroup/kogito"
+ DataIndexServiceName = "data-index-service"
+ DataIndexName = "data-index"
+ JobServiceName = "jobs-service"
+ ImageTag = "1.44"
+ PersistenceTypeEphemeral = "ephemeral"
+ PersistenceTypePostgressql = "postgresql"
microprofileServiceCatalogPropertyPrefix = "org.kie.kogito.addons.discovery."
discoveryLikePropertyPattern = "^\\${(kubernetes|knative|openshift):(.*)}$"
)
-var immutableApplicationProperties = "quarkus.http.port=" + DefaultHTTPWorkflowPortIntStr.String() + "\n" +
- "quarkus.http.host=0.0.0.0\n" +
- // We disable the Knative health checks to not block the dev pod to run if Knative objects are not available
- // See: https://kiegroup.github.io/kogito-docs/serverlessworkflow/latest/eventing/consume-produce-events-with-knative-eventing.html#ref-knative-eventing-add-on-source-configuration
- "org.kie.kogito.addons.knative.eventing.health-enabled=false\n" +
- "quarkus.devservices.enabled=false\n" +
- "quarkus.kogito.devservices.enabled=false\n"
+var (
+ immutableApplicationProperties = "quarkus.http.port=" + DefaultHTTPWorkflowPortIntStr.String() + "\n" +
+ "quarkus.http.host=0.0.0.0\n" +
+ // We disable the Knative health checks to not block the dev pod to run if Knative objects are not available
+ // See: https://kiegroup.github.io/kogito-docs/serverlessworkflow/latest/eventing/consume-produce-events-with-knative-eventing.html#ref-knative-eventing-add-on-source-configuration
+ "org.kie.kogito.addons.knative.eventing.health-enabled=false\n" +
+ "quarkus.devservices.enabled=false\n" +
+ "quarkus.kogito.devservices.enabled=false\n"
+
+ discoveryLikePropertyExpr = regexp.MustCompile(discoveryLikePropertyPattern)
+ _ AppPropertyHandler = &appPropertyHandler{}
+)
-var discoveryLikePropertyExpr = regexp.MustCompile(discoveryLikePropertyPattern)
+const (
+ DataIndexService = iota
+ JobService
+)
-var _ AppPropertyHandler = &appPropertyHandler{}
+// GetContainerName returns the name of the service's container in the deployment.
+func GetContainerName(stype int) string {
+ switch stype {
+ case DataIndexService:
+ return DataIndexServiceName
+ case JobService:
+ return JobServiceName
+ }
+ return ""
+}
+
+// GetServiceImageName returns the image name of the service's container. It takes in the service and persistence types and returns a string
+// that contains the FQDN of the image, including the tag.
+func GetServiceImageName(persistenceName string, stype int) string {
Review Comment:
Forgot to add the type. I'll push a new commit with changes from your comments and include the new type.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "tchughesiv (via GitHub)" <gi...@apache.org>.
tchughesiv commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1858378578
> If the jobs service is **NOT installed**, we still set the url, and we invent it :), to http://jobs-service/v2/jobs/events
This is dangerous as a service named `jobs-service` could exist within the namespace. are you sure this must be set? is there another, safer way to do this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1424842140
##########
controllers/profiles/common/constants/platform_services.go:
##########
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package constants
+
+const (
+ ConfigMapWorkflowPropsVolumeName = "workflow-properties"
+ PersistenceTypePostgreSQL = "postgresql"
+ PersistenceTypeEphemeral = "ephemeral"
+
+ DataIndexServiceURLProperty = "mp.messaging.outgoing.kogito-processinstances-events.url"
+ DataIndexServiceURLProtocol = "http"
+
+ JobServiceURLProperty = "mp.messaging.outgoing.kogito-job-service-job-request-events.url"
+ JobServiceKafkaSinkInjectionHealthCheck = `quarkus.smallrye-health.check."org.kie.kogito.jobs.service.messaging.http.health.knative.KSinkInjectionHealthCheck".enabled`
+ JobServiceStatusChangeEventsProperty = "kogito.jobs-service.http.job-status-change-events"
+ JobServiceStatusChangeEventsURL = "mp.messaging.outgoing.kogito-job-service-job-status-events-http.url"
+ JobServiceURLProtocol = "http"
+ JobServiceDataSourceReactiveURLProperty = "quarkus.datasource.reactive.url"
+
+ KogitoServiceUrlProperty = "kogito.service.url"
+ KogitoServiceUrlProtocol = "http"
+ KafkaSmallRyeHealthProperty = "quarkus.smallrye-health.check.\"io.quarkus.kafka.client.health.KafkaHealthCheck\".enabled"
+
+ MicroprofileServiceCatalogPropertyPrefix = "org.kie.kogito.addons.discovery."
+ DiscoveryLikePropertyPattern = "^\\${(kubernetes|knative|openshift):(.*)}$"
Review Comment:
Moved constant to the `workflows.go` file inside the constants package
##########
controllers/profiles/common/constants/platform_services.go:
##########
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package constants
+
+const (
+ ConfigMapWorkflowPropsVolumeName = "workflow-properties"
+ PersistenceTypePostgreSQL = "postgresql"
+ PersistenceTypeEphemeral = "ephemeral"
+
+ DataIndexServiceURLProperty = "mp.messaging.outgoing.kogito-processinstances-events.url"
+ DataIndexServiceURLProtocol = "http"
+
+ JobServiceURLProperty = "mp.messaging.outgoing.kogito-job-service-job-request-events.url"
+ JobServiceKafkaSinkInjectionHealthCheck = `quarkus.smallrye-health.check."org.kie.kogito.jobs.service.messaging.http.health.knative.KSinkInjectionHealthCheck".enabled`
+ JobServiceStatusChangeEventsProperty = "kogito.jobs-service.http.job-status-change-events"
+ JobServiceStatusChangeEventsURL = "mp.messaging.outgoing.kogito-job-service-job-status-events-http.url"
+ JobServiceURLProtocol = "http"
+ JobServiceDataSourceReactiveURLProperty = "quarkus.datasource.reactive.url"
+
+ KogitoServiceUrlProperty = "kogito.service.url"
+ KogitoServiceUrlProtocol = "http"
+ KafkaSmallRyeHealthProperty = "quarkus.smallrye-health.check.\"io.quarkus.kafka.client.health.KafkaHealthCheck\".enabled"
+
+ MicroprofileServiceCatalogPropertyPrefix = "org.kie.kogito.addons.discovery."
Review Comment:
Moved constant to the `workflows.go` file inside the constants package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "dmartinol (via GitHub)" <gi...@apache.org>.
dmartinol commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1427024129
##########
controllers/profiles/common/properties/application_test.go:
##########
@@ -238,3 +182,140 @@ func doTestGenerateMicroprofileServiceCatalogProperty(t *testing.T, serviceUri s
mpProperty := generateMicroprofileServiceCatalogProperty(serviceUri)
assert.Equal(t, mpProperty, expectedProperty, "expected microprofile service catalog property for serviceUri: %s, is %s, but the returned value was: %s", serviceUri, expectedProperty, mpProperty)
}
+
+func Test_appPropertyHandler_WithServicesWithUserOverrides(t *testing.T) {
+ //try to override kogito.service.url and quarkus.http.port
+ userProperties := "property1=value1\nproperty2=value2\nquarkus.http.port=9090\nkogito.service.url=http://myUrl.override.com\nquarkus.http.port=9090"
+ ns := "default"
+ workflow := test.GetBaseSonataFlow(ns)
+ workflow.SetAnnotations(map[string]string{metadata.Profile: string(metadata.DevProfile)})
+ enabled := true
+ platform := test.GetBasePlatform()
+ platform.Namespace = ns
+ platform.Spec = operatorapi.SonataFlowPlatformSpec{
+ Services: operatorapi.ServicesPlatformSpec{
+ DataIndex: &operatorapi.ServiceSpec{
+ Enabled: &enabled,
+ },
+ JobService: &operatorapi.ServiceSpec{
+ Enabled: &enabled,
+ },
+ },
+ }
+
+ di := services.NewDataIndexService(platform)
+
+ props, err := NewAppPropertyHandler(workflow, platform)
+ assert.NoError(t, err)
+ generatedProps, propsErr := properties.LoadString(props.WithUserProperties(userProperties).Build())
+ assert.NoError(t, propsErr)
+ assert.Equal(t, 8, len(generatedProps.Keys()))
+ assert.Equal(t, "value1", generatedProps.GetString("property1", ""))
+ assert.Equal(t, "value2", generatedProps.GetString("property2", ""))
+ //kogito.service.url takes the user provided value since it's a default mutable property.
+ assert.Equal(t, "http://myUrl.override.com", generatedProps.GetString("kogito.service.url", ""))
+ //quarkus.http.port remains with the default value since it's immutable.
+ assert.Equal(t, "8080", generatedProps.GetString("quarkus.http.port", ""))
+ assert.Equal(t, "0.0.0.0", generatedProps.GetString("quarkus.http.host", ""))
+ assert.Equal(t, "false", generatedProps.GetString("org.kie.kogito.addons.knative.eventing.health-enabled", ""))
+ assert.Equal(t, "false", generatedProps.GetString("quarkus.devservices.enabled", ""))
+ assert.Equal(t, "false", generatedProps.GetString("quarkus.kogito.devservices.enabled", ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.DataIndexServiceURLProperty, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceURLProperty, ""))
+
+ // prod profile enables config of outgoing events url
+ workflow.SetAnnotations(map[string]string{metadata.Profile: string(metadata.ProdProfile)})
+ props, err = NewAppPropertyHandler(workflow, platform)
+ assert.NoError(t, err)
+ generatedProps, propsErr = properties.LoadString(props.WithUserProperties(userProperties).Build())
+ assert.NoError(t, propsErr)
+ assert.Equal(t, 13, len(generatedProps.Keys()))
+ assert.Equal(t, "http://"+platform.Name+"-"+constants.DataIndexServiceName+"."+platform.Namespace+"/processes", generatedProps.GetString(constants.DataIndexServiceURLProperty, ""))
+ assert.Equal(t, "http://"+platform.Name+"-"+constants.JobServiceName+"."+platform.Namespace+"/v2/jobs/events", generatedProps.GetString(constants.JobServiceURLProperty, ""))
+ assert.Equal(t, "false", generatedProps.GetString(constants.JobServiceKafkaSinkInjectionHealthCheck, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceDataSourceReactiveURLProperty, ""))
+ assert.Equal(t, "true", generatedProps.GetString(constants.JobServiceStatusChangeEventsProperty, ""))
+ assert.Equal(t, generatedProps.GetString(constants.JobServiceStatusChangeEventsURL, ""), fmt.Sprintf("%s://%s.%s/jobs", constants.DataIndexServiceURLProtocol, di.GetServiceName(), platform.Namespace))
+
+ // disabling data index bypasses config of outgoing events url
+ platform.Spec.Services.DataIndex.Enabled = nil
+ props, err = NewAppPropertyHandler(workflow, platform)
+ assert.NoError(t, err)
+ generatedProps, propsErr = properties.LoadString(props.WithUserProperties(userProperties).Build())
+ assert.NoError(t, propsErr)
+ assert.Equal(t, 10, len(generatedProps.Keys()))
+ assert.Equal(t, "", generatedProps.GetString(constants.DataIndexServiceURLProperty, ""))
+ assert.Equal(t, "http://"+platform.Name+"-"+constants.JobServiceName+"."+platform.Namespace+"/v2/jobs/events", generatedProps.GetString(constants.JobServiceURLProperty, ""))
+ assert.Equal(t, "false", generatedProps.GetString(constants.JobServiceKafkaSinkInjectionHealthCheck, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceStatusChangeEventsProperty, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceStatusChangeEventsURL, ""))
+
+ // disabling job service bypasses config of outgoing events url
+ platform.Spec.Services.JobService.Enabled = nil
+ props, err = NewAppPropertyHandler(workflow, platform)
+ assert.NoError(t, err)
+ generatedProps, propsErr = properties.LoadString(props.WithUserProperties(userProperties).Build())
+ assert.NoError(t, propsErr)
+ assert.Equal(t, 8, len(generatedProps.Keys()))
+ assert.Equal(t, "", generatedProps.GetString(constants.DataIndexServiceURLProperty, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceURLProperty, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceKafkaSinkInjectionHealthCheck, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceDataSourceReactiveURLProperty, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceStatusChangeEventsProperty, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceStatusChangeEventsURL, ""))
+
+ // check that service app properties are being properly set
+ generatedProps, propsErr = properties.LoadString(NewServiceAppPropertyHandler(platform).WithUserProperties(userProperties).Build())
Review Comment:
in the business code (`services.go`) there is no such case where `NewServiceAppPropertyHandler` is decorated with `WithUserProperties`. Is this a correct population of the test data?
According to what I undertstood from @wmedvede
> there are no DI or JS properties that requires user intervention
so I would expect that there's no way to have `property1`, `property2` in the `generatedProps` (and `kogito.service.url` either), and the len should be 6 instead of 9
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "wmedvede (via GitHub)" <gi...@apache.org>.
wmedvede commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1424275229
##########
controllers/platform/services/properties.go:
##########
@@ -0,0 +1,98 @@
+// Copyright 2023 Apache Software Foundation (ASF)
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package services
+
+import (
+ "fmt"
+ "strings"
+
+ operatorapi "github.com/apache/incubator-kie-kogito-serverless-operator/api/v1alpha08"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles/common/constants"
+
+ "github.com/magiconair/properties"
+)
+
+func generateReactiveURL(postgresSpec *operatorapi.PersistencePostgreSql, schema string, namespace string, dbName string, port int) string {
+ if len(postgresSpec.JdbcUrl) > 0 && strings.Contains(postgresSpec.JdbcUrl, "currentSchema=") {
+ return strings.Replace(strings.TrimPrefix(postgresSpec.JdbcUrl, "jdbc:"), "currentSchema=", "search_path=", -1)
Review Comment:
be careful here, I believe this JdbcUrl case only works if "currentSchema" is present, however an url like this is still valid.
jdbc:postgresql://postgres:5432/sonataflow
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "tchughesiv (via GitHub)" <gi...@apache.org>.
tchughesiv commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1416038734
##########
controllers/platform/services.go:
##########
@@ -99,27 +98,9 @@ func createDataIndexDeployment(ctx context.Context, client client.Client, platfo
liveProbe := readyProbe.DeepCopy()
liveProbe.ProbeHandler.HTTPGet.Path = common.QuarkusHealthPathLive
dataDeployContainer := &corev1.Container{
Review Comment:
i think he means, why is `data` part of the `dataDeployContainer` var. with @jordigilh's other changes, this could maybe be changed to `deployContainer`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1840981681
@masayag PTAL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "tchughesiv (via GitHub)" <gi...@apache.org>.
tchughesiv commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1847194535
@jordigilh does your PR descrip still match the changes you're making here? if not, could you update it? thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "tchughesiv (via GitHub)" <gi...@apache.org>.
tchughesiv commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1427091996
##########
controllers/profiles/common/properties/application_test.go:
##########
@@ -238,3 +182,140 @@ func doTestGenerateMicroprofileServiceCatalogProperty(t *testing.T, serviceUri s
mpProperty := generateMicroprofileServiceCatalogProperty(serviceUri)
assert.Equal(t, mpProperty, expectedProperty, "expected microprofile service catalog property for serviceUri: %s, is %s, but the returned value was: %s", serviceUri, expectedProperty, mpProperty)
}
+
+func Test_appPropertyHandler_WithServicesWithUserOverrides(t *testing.T) {
+ //try to override kogito.service.url and quarkus.http.port
+ userProperties := "property1=value1\nproperty2=value2\nquarkus.http.port=9090\nkogito.service.url=http://myUrl.override.com\nquarkus.http.port=9090"
+ ns := "default"
+ workflow := test.GetBaseSonataFlow(ns)
+ workflow.SetAnnotations(map[string]string{metadata.Profile: string(metadata.DevProfile)})
+ enabled := true
+ platform := test.GetBasePlatform()
+ platform.Namespace = ns
+ platform.Spec = operatorapi.SonataFlowPlatformSpec{
+ Services: operatorapi.ServicesPlatformSpec{
+ DataIndex: &operatorapi.ServiceSpec{
+ Enabled: &enabled,
+ },
+ JobService: &operatorapi.ServiceSpec{
+ Enabled: &enabled,
+ },
+ },
+ }
+
+ di := services.NewDataIndexService(platform)
+
+ props, err := NewAppPropertyHandler(workflow, platform)
+ assert.NoError(t, err)
+ generatedProps, propsErr := properties.LoadString(props.WithUserProperties(userProperties).Build())
+ assert.NoError(t, propsErr)
+ assert.Equal(t, 8, len(generatedProps.Keys()))
+ assert.Equal(t, "value1", generatedProps.GetString("property1", ""))
+ assert.Equal(t, "value2", generatedProps.GetString("property2", ""))
+ //kogito.service.url takes the user provided value since it's a default mutable property.
+ assert.Equal(t, "http://myUrl.override.com", generatedProps.GetString("kogito.service.url", ""))
+ //quarkus.http.port remains with the default value since it's immutable.
+ assert.Equal(t, "8080", generatedProps.GetString("quarkus.http.port", ""))
+ assert.Equal(t, "0.0.0.0", generatedProps.GetString("quarkus.http.host", ""))
+ assert.Equal(t, "false", generatedProps.GetString("org.kie.kogito.addons.knative.eventing.health-enabled", ""))
+ assert.Equal(t, "false", generatedProps.GetString("quarkus.devservices.enabled", ""))
+ assert.Equal(t, "false", generatedProps.GetString("quarkus.kogito.devservices.enabled", ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.DataIndexServiceURLProperty, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceURLProperty, ""))
+
+ // prod profile enables config of outgoing events url
+ workflow.SetAnnotations(map[string]string{metadata.Profile: string(metadata.ProdProfile)})
+ props, err = NewAppPropertyHandler(workflow, platform)
+ assert.NoError(t, err)
+ generatedProps, propsErr = properties.LoadString(props.WithUserProperties(userProperties).Build())
+ assert.NoError(t, propsErr)
+ assert.Equal(t, 13, len(generatedProps.Keys()))
+ assert.Equal(t, "http://"+platform.Name+"-"+constants.DataIndexServiceName+"."+platform.Namespace+"/processes", generatedProps.GetString(constants.DataIndexServiceURLProperty, ""))
+ assert.Equal(t, "http://"+platform.Name+"-"+constants.JobServiceName+"."+platform.Namespace+"/v2/jobs/events", generatedProps.GetString(constants.JobServiceURLProperty, ""))
+ assert.Equal(t, "false", generatedProps.GetString(constants.JobServiceKafkaSinkInjectionHealthCheck, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceDataSourceReactiveURLProperty, ""))
+ assert.Equal(t, "true", generatedProps.GetString(constants.JobServiceStatusChangeEventsProperty, ""))
+ assert.Equal(t, generatedProps.GetString(constants.JobServiceStatusChangeEventsURL, ""), fmt.Sprintf("%s://%s.%s/jobs", constants.DataIndexServiceURLProtocol, di.GetServiceName(), platform.Namespace))
+
+ // disabling data index bypasses config of outgoing events url
+ platform.Spec.Services.DataIndex.Enabled = nil
+ props, err = NewAppPropertyHandler(workflow, platform)
+ assert.NoError(t, err)
+ generatedProps, propsErr = properties.LoadString(props.WithUserProperties(userProperties).Build())
+ assert.NoError(t, propsErr)
+ assert.Equal(t, 10, len(generatedProps.Keys()))
+ assert.Equal(t, "", generatedProps.GetString(constants.DataIndexServiceURLProperty, ""))
+ assert.Equal(t, "http://"+platform.Name+"-"+constants.JobServiceName+"."+platform.Namespace+"/v2/jobs/events", generatedProps.GetString(constants.JobServiceURLProperty, ""))
+ assert.Equal(t, "false", generatedProps.GetString(constants.JobServiceKafkaSinkInjectionHealthCheck, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceStatusChangeEventsProperty, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceStatusChangeEventsURL, ""))
+
+ // disabling job service bypasses config of outgoing events url
+ platform.Spec.Services.JobService.Enabled = nil
+ props, err = NewAppPropertyHandler(workflow, platform)
+ assert.NoError(t, err)
+ generatedProps, propsErr = properties.LoadString(props.WithUserProperties(userProperties).Build())
+ assert.NoError(t, propsErr)
+ assert.Equal(t, 8, len(generatedProps.Keys()))
+ assert.Equal(t, "", generatedProps.GetString(constants.DataIndexServiceURLProperty, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceURLProperty, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceKafkaSinkInjectionHealthCheck, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceDataSourceReactiveURLProperty, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceStatusChangeEventsProperty, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceStatusChangeEventsURL, ""))
+
+ // check that service app properties are being properly set
+ generatedProps, propsErr = properties.LoadString(NewServiceAppPropertyHandler(platform).WithUserProperties(userProperties).Build())
Review Comment:
> there are no DI or JS properties that requires user intervention
while there may not be properties that "require" user intervention... in my opinion, the operator shouldn't be in the business of limiting the user wrt property configs either. we already allow the user to modify just about everything in the DI & JS PodSpec... application properties should be no different. this kind of functionality allows for greater adoption and success in future production-like envs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "tchughesiv (via GitHub)" <gi...@apache.org>.
tchughesiv commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1860441339
> So your concern is valid, but we will have a problem deploying the jobs-service before getting there since the deployment name will already be taken. So we need to fix the naming clash while creating these objects.
@ricardozanini this is not an issue. objects can be identically named, as long as their `kind` is different. this already works for DI in the main branch and will work just as well for JS. this is actually ideal as the user has better understanding of which objects are interconnected. it's easier for operator to maintain as well.
that said, my concern is still valid, but for a different reason. we should only use something that will resolve "locally" to the container itself. e.g. localhost, 127.0.0.1, etc. Just because the operator hasn't created a `job-service` service, this doesn't mean a user hasn't done so manually.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "tchughesiv (via GitHub)" <gi...@apache.org>.
tchughesiv commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1422792873
##########
controllers/platform/services.go:
##########
@@ -58,30 +55,32 @@ func (action *serviceAction) Handle(ctx context.Context, platform *operatorapi.S
return nil, err
}
- if err := createDataIndexComponents(ctx, action.client, platform); err != nil {
- return nil, err
+ if platform.Spec.Services.DataIndex != nil {
Review Comment:
no, we will not replicate the spec in cluster-scope. we previously discussed this and decided to only reference the chosen platform in spec and leave it at that. the referenced SonataFlowPlatform will be the single source of truth.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "ricardozanini (via GitHub)" <gi...@apache.org>.
ricardozanini merged PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "ricardozanini (via GitHub)" <gi...@apache.org>.
ricardozanini commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1417300076
##########
controllers/platform/services.go:
##########
@@ -58,30 +55,32 @@ func (action *serviceAction) Handle(ctx context.Context, platform *operatorapi.S
return nil, err
}
- if err := createDataIndexComponents(ctx, action.client, platform); err != nil {
- return nil, err
+ if platform.Spec.Services.DataIndex != nil {
Review Comment:
I agree, once we have configs changed, we then rollout every workflow removing this property so then we scale it to 0. For this initial version, is fine the way it is.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "ricardozanini (via GitHub)" <gi...@apache.org>.
ricardozanini commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1422734247
##########
controllers/platform/services.go:
##########
@@ -58,30 +55,32 @@ func (action *serviceAction) Handle(ctx context.Context, platform *operatorapi.S
return nil, err
}
- if err := createDataIndexComponents(ctx, action.client, platform); err != nil {
- return nil, err
+ if platform.Spec.Services.DataIndex != nil {
Review Comment:
@masayag that sounds reasonable, mind opening an issue here describing this behavior so we can discuss it more in-depth? We might have to implement it preferably before the cluster scope one. @tchughesiv wdyt about his proposal?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "masayag (via GitHub)" <gi...@apache.org>.
masayag commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1422820576
##########
controllers/platform/services.go:
##########
@@ -58,30 +55,32 @@ func (action *serviceAction) Handle(ctx context.Context, platform *operatorapi.S
return nil, err
}
- if err := createDataIndexComponents(ctx, action.client, platform); err != nil {
- return nil, err
+ if platform.Spec.Services.DataIndex != nil {
Review Comment:
See https://github.com/apache/incubator-kie-kogito-serverless-operator/issues/321
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1426124264
##########
controllers/profiles/common/properties/application.go:
##########
@@ -143,31 +131,18 @@ func (a *appPropertyHandler) Build() string {
func (a *appPropertyHandler) withKogitoServiceUrl() AppPropertyHandler {
var kogitoServiceUrl string
if len(a.workflow.Namespace) > 0 {
- kogitoServiceUrl = fmt.Sprintf("%s://%s.%s", kogitoServiceUrlProtocol, a.workflow.Name, a.workflow.Namespace)
+ kogitoServiceUrl = fmt.Sprintf("%s://%s.%s", constants.KogitoServiceUrlProtocol, a.workflow.Name, a.workflow.Namespace)
} else {
- kogitoServiceUrl = fmt.Sprintf("%s://%s", kogitoServiceUrlProtocol, a.workflow.Name)
+ kogitoServiceUrl = fmt.Sprintf("%s://%s", constants.KogitoServiceUrlProtocol, a.workflow.Name)
}
- return a.addDefaultMutableProperty(kogitoServiceUrlProperty, kogitoServiceUrl)
-}
-
-// withDataIndexServiceUrl adds the property dataIndexServiceUrlProperty to the application properties.
-// See Service Discovery https://kubernetes.io/docs/concepts/services-networking/service/#dns
-func (a *appPropertyHandler) withDataIndexServiceUrl() AppPropertyHandler {
- if profiles.IsProdProfile(a.workflow) && dataIndexEnabled(a.platform) {
Review Comment:
Done to check if the workflow does not contain the `Dev` profile instead of checking if it has the `Prod` profile
##########
controllers/platform/services/properties.go:
##########
@@ -0,0 +1,131 @@
+// Copyright 2023 Apache Software Foundation (ASF)
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package services
+
+import (
+ "fmt"
+ "net/url"
+ "strings"
+
+ operatorapi "github.com/apache/incubator-kie-kogito-serverless-operator/api/v1alpha08"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles/common/constants"
+
+ "github.com/magiconair/properties"
+)
+
+func generateReactiveURL(postgresSpec *operatorapi.PersistencePostgreSql, schema string, namespace string, dbName string, port int) (string, error) {
+ if len(postgresSpec.JdbcUrl) > 0 {
+ s := strings.TrimLeft(postgresSpec.JdbcUrl, "jdbc:")
+ u, err := url.Parse(s)
+ if err != nil {
+ return "", err
+ }
+ ret := fmt.Sprintf("%s://", u.Scheme)
+ if len(u.User.Username()) > 0 {
+ p, ok := u.User.Password()
+ if ok {
+ ret = fmt.Sprintf("%s%s:%s@", ret, u.User.Username(), p)
+ }
+ }
+ ret = fmt.Sprintf("%s%s%s?", ret, u.Host, u.Path)
+ kv, err := url.ParseQuery(u.RawQuery)
+ if err != nil {
+ return "", err
+ }
+ spv := schema
+ if v, ok := kv["search_path"]; ok {
+ for _, val := range v {
+ if len(val) != 0 {
+ spv = v[0]
+ }
+ }
+ } else if v, ok := kv["currentSchema"]; ok {
+ for _, val := range v {
+ if len(val) != 0 {
+ spv = v[0]
+ }
+ }
+ }
+ return fmt.Sprintf("%ssearch_path=%s", ret, spv), nil
Review Comment:
Done. PTAL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "tchughesiv (via GitHub)" <gi...@apache.org>.
tchughesiv commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1830672439
lgtm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "ricardozanini (via GitHub)" <gi...@apache.org>.
ricardozanini commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1408252335
##########
controllers/profiles/common/app_properties.go:
##########
@@ -45,27 +45,59 @@ const (
kafkaSmallRyeHealthProperty = "quarkus.smallrye-health.check.\"io.quarkus.kafka.client.health.KafkaHealthCheck\".enabled"
dataIndexServiceUrlProtocol = "http"
- DataIndexImageBase = "quay.io/kiegroup/kogito-data-index-"
- DataIndexName = "data-index-service"
-
- PersistenceTypeEphemeral = "ephemeral"
- PersistenceTypePostgressql = "postgresql"
-
+ imageNamePrefix = "quay.io/kiegroup/kogito"
+ DataIndexServiceName = "data-index-service"
+ DataIndexName = "data-index"
+ JobServiceName = "jobs-service"
+ ImageTag = "1.44"
Review Comment:
Please use this instead: https://github.com/apache/incubator-kie-kogito-serverless-operator/blob/main/version/version.go#L20
The `main` branch must use the nightly images: https://quay.io/repository/kiegroup/kogito-jobs-service-postgresql-nightly?tab=tags
So if `version.IsSnapshot()`, we use a nightly image version with `latest` tag, otherwise `version.GetMajorMinor()`.
Our CI updates that file when branching and releasing, hence we must not fix the version in the code base.
##########
controllers/profiles/common/app_properties.go:
##########
@@ -45,27 +45,59 @@ const (
kafkaSmallRyeHealthProperty = "quarkus.smallrye-health.check.\"io.quarkus.kafka.client.health.KafkaHealthCheck\".enabled"
dataIndexServiceUrlProtocol = "http"
- DataIndexImageBase = "quay.io/kiegroup/kogito-data-index-"
- DataIndexName = "data-index-service"
-
- PersistenceTypeEphemeral = "ephemeral"
- PersistenceTypePostgressql = "postgresql"
-
+ imageNamePrefix = "quay.io/kiegroup/kogito"
+ DataIndexServiceName = "data-index-service"
+ DataIndexName = "data-index"
+ JobServiceName = "jobs-service"
+ ImageTag = "1.44"
+ PersistenceTypeEphemeral = "ephemeral"
+ PersistenceTypePostgressql = "postgresql"
microprofileServiceCatalogPropertyPrefix = "org.kie.kogito.addons.discovery."
discoveryLikePropertyPattern = "^\\${(kubernetes|knative|openshift):(.*)}$"
)
-var immutableApplicationProperties = "quarkus.http.port=" + DefaultHTTPWorkflowPortIntStr.String() + "\n" +
- "quarkus.http.host=0.0.0.0\n" +
- // We disable the Knative health checks to not block the dev pod to run if Knative objects are not available
- // See: https://kiegroup.github.io/kogito-docs/serverlessworkflow/latest/eventing/consume-produce-events-with-knative-eventing.html#ref-knative-eventing-add-on-source-configuration
- "org.kie.kogito.addons.knative.eventing.health-enabled=false\n" +
- "quarkus.devservices.enabled=false\n" +
- "quarkus.kogito.devservices.enabled=false\n"
+var (
+ immutableApplicationProperties = "quarkus.http.port=" + DefaultHTTPWorkflowPortIntStr.String() + "\n" +
+ "quarkus.http.host=0.0.0.0\n" +
+ // We disable the Knative health checks to not block the dev pod to run if Knative objects are not available
+ // See: https://kiegroup.github.io/kogito-docs/serverlessworkflow/latest/eventing/consume-produce-events-with-knative-eventing.html#ref-knative-eventing-add-on-source-configuration
+ "org.kie.kogito.addons.knative.eventing.health-enabled=false\n" +
+ "quarkus.devservices.enabled=false\n" +
+ "quarkus.kogito.devservices.enabled=false\n"
+
+ discoveryLikePropertyExpr = regexp.MustCompile(discoveryLikePropertyPattern)
+ _ AppPropertyHandler = &appPropertyHandler{}
+)
-var discoveryLikePropertyExpr = regexp.MustCompile(discoveryLikePropertyPattern)
+const (
+ DataIndexService = iota
+ JobService
+)
-var _ AppPropertyHandler = &appPropertyHandler{}
+// GetContainerName returns the name of the service's container in the deployment.
+func GetContainerName(stype int) string {
+ switch stype {
+ case DataIndexService:
+ return DataIndexServiceName
+ case JobService:
+ return JobServiceName
+ }
+ return ""
+}
+
+// GetServiceImageName returns the image name of the service's container. It takes in the service and persistence types and returns a string
+// that contains the FQDN of the image, including the tag.
+func GetServiceImageName(persistenceName string, stype int) string {
Review Comment:
Same question here.
##########
controllers/profiles/common/app_properties.go:
##########
@@ -45,27 +45,59 @@ const (
kafkaSmallRyeHealthProperty = "quarkus.smallrye-health.check.\"io.quarkus.kafka.client.health.KafkaHealthCheck\".enabled"
dataIndexServiceUrlProtocol = "http"
- DataIndexImageBase = "quay.io/kiegroup/kogito-data-index-"
- DataIndexName = "data-index-service"
-
- PersistenceTypeEphemeral = "ephemeral"
- PersistenceTypePostgressql = "postgresql"
-
+ imageNamePrefix = "quay.io/kiegroup/kogito"
+ DataIndexServiceName = "data-index-service"
+ DataIndexName = "data-index"
+ JobServiceName = "jobs-service"
+ ImageTag = "1.44"
+ PersistenceTypeEphemeral = "ephemeral"
+ PersistenceTypePostgressql = "postgresql"
microprofileServiceCatalogPropertyPrefix = "org.kie.kogito.addons.discovery."
discoveryLikePropertyPattern = "^\\${(kubernetes|knative|openshift):(.*)}$"
)
-var immutableApplicationProperties = "quarkus.http.port=" + DefaultHTTPWorkflowPortIntStr.String() + "\n" +
- "quarkus.http.host=0.0.0.0\n" +
- // We disable the Knative health checks to not block the dev pod to run if Knative objects are not available
- // See: https://kiegroup.github.io/kogito-docs/serverlessworkflow/latest/eventing/consume-produce-events-with-knative-eventing.html#ref-knative-eventing-add-on-source-configuration
- "org.kie.kogito.addons.knative.eventing.health-enabled=false\n" +
- "quarkus.devservices.enabled=false\n" +
- "quarkus.kogito.devservices.enabled=false\n"
+var (
+ immutableApplicationProperties = "quarkus.http.port=" + DefaultHTTPWorkflowPortIntStr.String() + "\n" +
+ "quarkus.http.host=0.0.0.0\n" +
+ // We disable the Knative health checks to not block the dev pod to run if Knative objects are not available
+ // See: https://kiegroup.github.io/kogito-docs/serverlessworkflow/latest/eventing/consume-produce-events-with-knative-eventing.html#ref-knative-eventing-add-on-source-configuration
+ "org.kie.kogito.addons.knative.eventing.health-enabled=false\n" +
+ "quarkus.devservices.enabled=false\n" +
+ "quarkus.kogito.devservices.enabled=false\n"
+
+ discoveryLikePropertyExpr = regexp.MustCompile(discoveryLikePropertyPattern)
+ _ AppPropertyHandler = &appPropertyHandler{}
+)
-var discoveryLikePropertyExpr = regexp.MustCompile(discoveryLikePropertyPattern)
+const (
+ DataIndexService = iota
+ JobService
+)
-var _ AppPropertyHandler = &appPropertyHandler{}
+// GetContainerName returns the name of the service's container in the deployment.
+func GetContainerName(stype int) string {
Review Comment:
Can you please type this enum? Is this function required to be exported?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1409417409
##########
controllers/profiles/common/app_properties_test.go:
##########
@@ -128,7 +128,7 @@ func Test_appPropertyHandler_WithServicesWithUserOverrides(t *testing.T) {
generatedProps, propsErr = properties.LoadString(props)
assert.NoError(t, propsErr)
assert.Equal(t, 9, len(generatedProps.Keys()))
- assert.Equal(t, "http://"+platform.Name+"-"+DataIndexName+"."+platform.Namespace+"/processes", generatedProps.GetString(dataIndexServiceUrlProperty, ""))
+ assert.Equal(t, "http://"+platform.Name+"-"+DataIndexServiceName+"."+platform.Namespace+"/processes", generatedProps.GetString(dataIndexServiceUrlProperty, ""))
Review Comment:
Done in the same test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "wmedvede (via GitHub)" <gi...@apache.org>.
wmedvede commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1410551982
##########
controllers/profiles/common/app_properties.go:
##########
@@ -44,28 +45,67 @@ const (
dataIndexServiceUrlProperty = "mp.messaging.outgoing.kogito-processinstances-events.url"
kafkaSmallRyeHealthProperty = "quarkus.smallrye-health.check.\"io.quarkus.kafka.client.health.KafkaHealthCheck\".enabled"
dataIndexServiceUrlProtocol = "http"
-
- DataIndexImageBase = "quay.io/kiegroup/kogito-data-index-"
- DataIndexName = "data-index-service"
-
- PersistenceTypeEphemeral = "ephemeral"
- PersistenceTypePostgressql = "postgresql"
-
+ jobServiceURLProperty = "mp.messaging.outgoing.kogito-job-service-job-request-events.url"
+ jobServiceURLProtocol = "http"
+
+ imageNamePrefix = "quay.io/kiegroup/kogito"
+ DataIndexServiceName = "data-index-service"
+ DataIndexName = "data-index"
+ JobServiceName = "jobs-service"
+ PersistenceTypeEphemeral = "ephemeral"
+ PersistenceTypePostgressql = "postgresql"
microprofileServiceCatalogPropertyPrefix = "org.kie.kogito.addons.discovery."
discoveryLikePropertyPattern = "^\\${(kubernetes|knative|openshift):(.*)}$"
)
-var immutableApplicationProperties = "quarkus.http.port=" + DefaultHTTPWorkflowPortIntStr.String() + "\n" +
- "quarkus.http.host=0.0.0.0\n" +
- // We disable the Knative health checks to not block the dev pod to run if Knative objects are not available
- // See: https://kiegroup.github.io/kogito-docs/serverlessworkflow/latest/eventing/consume-produce-events-with-knative-eventing.html#ref-knative-eventing-add-on-source-configuration
- "org.kie.kogito.addons.knative.eventing.health-enabled=false\n" +
- "quarkus.devservices.enabled=false\n" +
- "quarkus.kogito.devservices.enabled=false\n"
+var (
+ immutableApplicationProperties = "quarkus.http.port=" + DefaultHTTPWorkflowPortIntStr.String() + "\n" +
+ "quarkus.http.host=0.0.0.0\n" +
+ // We disable the Knative health checks to not block the dev pod to run if Knative objects are not available
+ // See: https://kiegroup.github.io/kogito-docs/serverlessworkflow/latest/eventing/consume-produce-events-with-knative-eventing.html#ref-knative-eventing-add-on-source-configuration
+ "org.kie.kogito.addons.knative.eventing.health-enabled=false\n" +
+ "quarkus.devservices.enabled=false\n" +
+ "quarkus.kogito.devservices.enabled=false\n"
+
+ discoveryLikePropertyExpr = regexp.MustCompile(discoveryLikePropertyPattern)
+ _ AppPropertyHandler = &appPropertyHandler{}
+)
-var discoveryLikePropertyExpr = regexp.MustCompile(discoveryLikePropertyPattern)
+type ServiceType int
-var _ AppPropertyHandler = &appPropertyHandler{}
+const (
+ DataIndexService ServiceType = iota
+ JobService
+)
+
+// GetContainerName returns the name of the service's container in the deployment.
+func GetContainerName(stype ServiceType) string {
+ switch stype {
+ case DataIndexService:
+ return DataIndexServiceName
+ case JobService:
+ return JobServiceName
+ }
+ return ""
+}
+
+// GetServiceImageName returns the image name of the service's container. It takes in the service and persistence types and returns a string
+// that contains the FQDN of the image, including the tag.
+func GetServiceImageName(persistenceName string, stype ServiceType) string {
Review Comment:
Let me add guys a general comment regarding the app_properties.go. I think that here, we originally started to manage the application.properties related to the workflows, but the application.properties related to the services are a different stuff. Now I think this is starting to become a sort of all in one.... Not sure it's good.
now we have things like this:
https://github.com/apache/incubator-kie-kogito-serverless-operator/blob/9a1e05d3e67081a9e2a713316abc7f60f3736a3d/controllers/profiles/common/app_properties.go#L256
with a common.NewServiceAppPropertyHandler that uses the original handler := &appPropertyHandler{
platform: platform,
isService: true,
}
and that we use when we need to create the applicaton.properties for a service, in places like this:
https://github.com/apache/incubator-kie-kogito-serverless-operator/blob/b017c4b41431772dbc398a51ddc12119cc11e4df/controllers/platform/services.go#L404
IDK, just thinking aloud, but maybe this deserves a piece of refactoring.
##########
controllers/platform/services.go:
##########
@@ -58,30 +60,72 @@ func (action *serviceAction) Handle(ctx context.Context, platform *operatorapi.S
return nil, err
}
- if err := createDataIndexComponents(ctx, action.client, platform); err != nil {
- return nil, err
+ if platform.Spec.Services.DataIndex != nil {
Review Comment:
I have question here, maybe I'm missing something or not understanding well, but my understanding is that when we execute this Handle, is because the platform.Status.IsReady().
So, when the platform is ready, we start to create the services. All good, but on the other hand, if the platform is ready, the workflows might have been started the creation procedure too.
if worklfows and services re being created at the same time, what happens if a workflow needs a service?
does it make sense?
return platform.Status.IsReady()
##########
controllers/profiles/common/app_properties.go:
##########
@@ -156,7 +194,7 @@ func (a *appPropertyHandler) withDataIndexServiceUrl() AppPropertyHandler {
if profiles.IsProdProfile(a.workflow) && dataIndexEnabled(a.platform) {
a.addDefaultMutableProperty(
dataIndexServiceUrlProperty,
- fmt.Sprintf("%s://%s.%s/processes", dataIndexServiceUrlProtocol, GetDataIndexName(a.platform), a.platform.Namespace),
+ fmt.Sprintf("%s://%s.%s/processes", dataIndexServiceUrlProtocol, GetServiceName(a.platform.Name, DataIndexService), a.platform.Namespace),
Review Comment:
yes, this mp.messaging.outgoing.kogito-job-service-job-request-events.url should be enough
##########
controllers/platform/services.go:
##########
@@ -99,7 +143,7 @@ func createDataIndexDeployment(ctx context.Context, client client.Client, platfo
liveProbe := readyProbe.DeepCopy()
liveProbe.ProbeHandler.HTTPGet.Path = common.QuarkusHealthPathLive
dataDeployContainer := &corev1.Container{
- Image: common.DataIndexImageBase + common.PersistenceTypeEphemeral,
+ Image: common.GetServiceImageName(common.PersistenceTypeEphemeral, serviceType),
Env: []corev1.EnvVar{
{
Review Comment:
This KOGITO_DATA_INDEX_QUARKUS_PROFILE is specific for the data-index, I think that here we should probably generate different env vars depending on the service type.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "masayag (via GitHub)" <gi...@apache.org>.
masayag commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1416069711
##########
controllers/platform/services.go:
##########
@@ -58,30 +55,32 @@ func (action *serviceAction) Handle(ctx context.Context, platform *operatorapi.S
return nil, err
}
- if err := createDataIndexComponents(ctx, action.client, platform); err != nil {
- return nil, err
+ if platform.Spec.Services.DataIndex != nil {
Review Comment:
My question is when would `platform.spec.services.dataIndex.enabled` be set to `false`?
Which business use case is served by disabling the data-index (or job-service) for prod workflows?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "ricardozanini (via GitHub)" <gi...@apache.org>.
ricardozanini commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1856597741
> The job service app properties are not added because the code that adds these properties is never called. The code is executed only when the a sonataflow CR is created, but not when the service is created. In this case should it add the same properties to the configmap as it does for a sonataflow CR? Or should it be a subset of those (reactiveURL and something else?).
1. When deploying Jobs Service or Data Index, the `SonataFlowPlatform` controller must set the config properties for each service accordingly for them to work independently.
2. During the `SonataFlow` controller reconciliation, if there's a `SonataFlowPlatform` with Jobs Service and DI available in the current context, it should have the workflow application properties being updated to add the required properties for the workflow to connect and send events to those services.
That's the use case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "tchughesiv (via GitHub)" <gi...@apache.org>.
tchughesiv commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1858103784
> > ```
> > 1. When deploying Jobs Service or Data Index, the `SonataFlowPlatform` controller must set the config properties for each service accordingly for them to work independently.
> > 2. During the `SonataFlow` controller reconciliation, if the workflow is not `dev` and if there's a `SonataFlowPlatform` with Jobs Service and DI available in the current context, it should have the workflow application properties being updated to add the required properties for the workflow to connect and send events to those services.
> > ```
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > That's the use case.
>
> Should the properties in each service be any different for the service itself or for the deployment triggered by a sonataflow CR instance? In other words: the properties specific for creating the job service are all included in the properties for a sonataflow CR? or a subset of those?
>
> Right now I consider them to be the same.
@jordigilh service properties will be different than workflow properties. sure, they may share a few similar configs but there will be configs that only services use... and configs that only the workflows will use. you should be able to reference what was already done w/ DI vs the workflow properties.. and then modify to fit the needs of job service. e.g. the `NewServiceAppPropertyHandler`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "ricardozanini (via GitHub)" <gi...@apache.org>.
ricardozanini commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1424396104
##########
controllers/platform/services/services.go:
##########
@@ -0,0 +1,298 @@
+// Copyright 2023 Red Hat, Inc. and/or its affiliates
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package services
+
+import (
+ "fmt"
+ "strconv"
+
+ corev1 "k8s.io/api/core/v1"
+ "k8s.io/apimachinery/pkg/api/resource"
+
+ operatorapi "github.com/apache/incubator-kie-kogito-serverless-operator/api/v1alpha08"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles/common/constants"
+
+ "github.com/apache/incubator-kie-kogito-serverless-operator/version"
+ "github.com/imdario/mergo"
+)
+
+type Platform interface {
+ // GetContainerName returns the name of the service's container in the deployment.
+ GetContainerName() string
+ // GetServiceImageName returns the image name of the service's container. It takes in the service and persistence types and returns a string
+ // that contains the FQDN of the image, including the tag.
+ GetServiceImageName(persistenceName string) string
+ // GetServiceName returns the name of the kubernetes service prefixed with the platform name
+ GetServiceName() string
+ // GetServiceCmName returns the name of the configmap associated to the service
+ GetServiceCmName() string
+ // GetEnvironmentVariables returns the env variables to be injected to the service container
+ GetEnvironmentVariables() []corev1.EnvVar
+ // GetResourceLimits returns the pod's memory and CPU resource requirements
+ // Values for job service taken from
+ // https://github.com/parodos-dev/orchestrator-helm-chart/blob/52d09eda56fdbed3060782df29847c97f172600f/charts/orchestrator/values.yaml#L68-L72
+ GetPodResourceRequirements() corev1.ResourceRequirements
+ // GetReplicaCountForService Returns the default pod replica count for the given service
+ GetReplicaCount() int32
Review Comment:
@wmedvede well noted, we can't scale Jobs Service to more than one replica.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1424842140
##########
controllers/profiles/common/constants/platform_services.go:
##########
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package constants
+
+const (
+ ConfigMapWorkflowPropsVolumeName = "workflow-properties"
+ PersistenceTypePostgreSQL = "postgresql"
+ PersistenceTypeEphemeral = "ephemeral"
+
+ DataIndexServiceURLProperty = "mp.messaging.outgoing.kogito-processinstances-events.url"
+ DataIndexServiceURLProtocol = "http"
+
+ JobServiceURLProperty = "mp.messaging.outgoing.kogito-job-service-job-request-events.url"
+ JobServiceKafkaSinkInjectionHealthCheck = `quarkus.smallrye-health.check."org.kie.kogito.jobs.service.messaging.http.health.knative.KSinkInjectionHealthCheck".enabled`
+ JobServiceStatusChangeEventsProperty = "kogito.jobs-service.http.job-status-change-events"
+ JobServiceStatusChangeEventsURL = "mp.messaging.outgoing.kogito-job-service-job-status-events-http.url"
+ JobServiceURLProtocol = "http"
+ JobServiceDataSourceReactiveURLProperty = "quarkus.datasource.reactive.url"
+
+ KogitoServiceUrlProperty = "kogito.service.url"
+ KogitoServiceUrlProtocol = "http"
+ KafkaSmallRyeHealthProperty = "quarkus.smallrye-health.check.\"io.quarkus.kafka.client.health.KafkaHealthCheck\".enabled"
+
+ MicroprofileServiceCatalogPropertyPrefix = "org.kie.kogito.addons.discovery."
+ DiscoveryLikePropertyPattern = "^\\${(kubernetes|knative|openshift):(.*)}$"
Review Comment:
Moved constant to the `workflows.go` file inside the constants package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "tchughesiv (via GitHub)" <gi...@apache.org>.
tchughesiv commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1425318651
##########
controllers/profiles/common/properties/application.go:
##########
@@ -143,31 +131,18 @@ func (a *appPropertyHandler) Build() string {
func (a *appPropertyHandler) withKogitoServiceUrl() AppPropertyHandler {
var kogitoServiceUrl string
if len(a.workflow.Namespace) > 0 {
- kogitoServiceUrl = fmt.Sprintf("%s://%s.%s", kogitoServiceUrlProtocol, a.workflow.Name, a.workflow.Namespace)
+ kogitoServiceUrl = fmt.Sprintf("%s://%s.%s", constants.KogitoServiceUrlProtocol, a.workflow.Name, a.workflow.Namespace)
} else {
- kogitoServiceUrl = fmt.Sprintf("%s://%s", kogitoServiceUrlProtocol, a.workflow.Name)
+ kogitoServiceUrl = fmt.Sprintf("%s://%s", constants.KogitoServiceUrlProtocol, a.workflow.Name)
}
- return a.addDefaultMutableProperty(kogitoServiceUrlProperty, kogitoServiceUrl)
-}
-
-// withDataIndexServiceUrl adds the property dataIndexServiceUrlProperty to the application properties.
-// See Service Discovery https://kubernetes.io/docs/concepts/services-networking/service/#dns
-func (a *appPropertyHandler) withDataIndexServiceUrl() AppPropertyHandler {
- if profiles.IsProdProfile(a.workflow) && dataIndexEnabled(a.platform) {
Review Comment:
if we're now configuring ALL workflows with these services, then many of api descriptions and references to this functionality would need to be changed as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1425822412
##########
controllers/platform/services/properties.go:
##########
@@ -0,0 +1,131 @@
+// Copyright 2023 Apache Software Foundation (ASF)
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package services
+
+import (
+ "fmt"
+ "net/url"
+ "strings"
+
+ operatorapi "github.com/apache/incubator-kie-kogito-serverless-operator/api/v1alpha08"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles/common/constants"
+
+ "github.com/magiconair/properties"
+)
+
+func generateReactiveURL(postgresSpec *operatorapi.PersistencePostgreSql, schema string, namespace string, dbName string, port int) (string, error) {
+ if len(postgresSpec.JdbcUrl) > 0 {
+ s := strings.TrimLeft(postgresSpec.JdbcUrl, "jdbc:")
+ u, err := url.Parse(s)
+ if err != nil {
+ return "", err
+ }
+ ret := fmt.Sprintf("%s://", u.Scheme)
+ if len(u.User.Username()) > 0 {
+ p, ok := u.User.Password()
+ if ok {
+ ret = fmt.Sprintf("%s%s:%s@", ret, u.User.Username(), p)
+ }
+ }
+ ret = fmt.Sprintf("%s%s%s?", ret, u.Host, u.Path)
+ kv, err := url.ParseQuery(u.RawQuery)
+ if err != nil {
+ return "", err
+ }
+ spv := schema
Review Comment:
When the schema is not found in the URL, is that OK to return the JDBC URL without any parameter?
Example:
Source URL `jdbc:postgres://host.com:5432/path?k=v#f` -> Reactive URL ->`postgres://host.com:5432/path`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "tchughesiv (via GitHub)" <gi...@apache.org>.
tchughesiv commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1425298412
##########
controllers/platform/services/properties.go:
##########
@@ -0,0 +1,98 @@
+// Copyright 2023 Apache Software Foundation (ASF)
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package services
+
+import (
+ "fmt"
+ "strings"
+
+ operatorapi "github.com/apache/incubator-kie-kogito-serverless-operator/api/v1alpha08"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles/common/constants"
+
+ "github.com/magiconair/properties"
+)
+
+func generateReactiveURL(postgresSpec *operatorapi.PersistencePostgreSql, schema string, namespace string, dbName string, port int) string {
+ if len(postgresSpec.JdbcUrl) > 0 && strings.Contains(postgresSpec.JdbcUrl, "currentSchema=") {
+ return strings.Replace(strings.TrimPrefix(postgresSpec.JdbcUrl, "jdbc:"), "currentSchema=", "search_path=", -1)
+ }
+ databaseSchema := schema
+ if len(postgresSpec.ServiceRef.DatabaseSchema) > 0 {
+ databaseSchema = postgresSpec.ServiceRef.DatabaseSchema
+ }
+ databaseNamespace := namespace
+ if len(postgresSpec.ServiceRef.Namespace) > 0 {
+ databaseNamespace = postgresSpec.ServiceRef.Namespace
+ }
+ dataSourcePort := port
+ if postgresSpec.ServiceRef.Port != nil {
+ dataSourcePort = *postgresSpec.ServiceRef.Port
+ }
+ databaseName := dbName
+ if len(postgresSpec.ServiceRef.DatabaseName) > 0 {
+ databaseName = postgresSpec.ServiceRef.DatabaseName
+ }
+ return fmt.Sprintf("%s://%s:%d/%s?search_path=%s", constants.PersistenceTypePostgreSQL, postgresSpec.ServiceRef.Name+"."+databaseNamespace, dataSourcePort, databaseName, databaseSchema)
+}
+
+// GenerateDataIndexApplicationProperties returns the application properties required for the Data Index Service to work when deployed in a production profile
+// and data index' service's spec field `Enabled` set to true
+func GenerateDataIndexApplicationProperties(workflow *operatorapi.SonataFlow, platform *operatorapi.SonataFlowPlatform) *properties.Properties {
+ props := properties.NewProperties()
+ if profiles.IsProdProfile(workflow) && dataIndexEnabled(platform) {
+ di := NewDataIndexService(platform)
+ props.Set(
+ constants.DataIndexServiceURLProperty,
+ fmt.Sprintf("%s://%s.%s/processes", constants.DataIndexServiceURLProtocol, di.GetServiceName(), platform.Namespace),
+ )
+ }
+ return props
+}
+
+// GenerateJobServiceApplicationProperties returns the application properties required for the Job Service to work in a production profile and job service's
+// service's spec field `Enabled` set to true
+func GenerateJobServiceApplicationProperties(workflow *operatorapi.SonataFlow, platform *operatorapi.SonataFlowPlatform) *properties.Properties {
+ props := properties.NewProperties()
+ if profiles.IsProdProfile(workflow) && jobServiceEnabled(platform) {
Review Comment:
@jordigilh can you explain what was changed here and why?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "ricardozanini (via GitHub)" <gi...@apache.org>.
ricardozanini commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1860207691
> > If the jobs service is **NOT installed**, we still set the url, and we invent it :), to http://jobs-service/v2/jobs/events
>
> @wmedvede This is dangerous as a service named `jobs-service` could exist within the namespace. are you sure this must be set? is there a different, safer way to do this? maybe `http://localhost/v2/jobs/events` ?
The service URL is based on the objects we created in the namespace. Given that the default naming of the Deployment and Service are `jobs-service`, the DNS will resolve to `jobs-service.svc.cluster.local`, correct?
So your concern is valid, but we will have a problem deploying the jobs-service before getting there since the deployment name will already be taken. So we need to fix the naming clash while creating these objects.
We can table this discussion for the moment and open a follow-up task to deal with the naming clash for our managed support services.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1862120550
> ./local/hack/run-e2e.
Thanks! but I ended up running just `make test-e2e` and it worked 😄 .
Based on what I can gather from the pod's logs, it's failing due to an exception at startup:
```java
orderprocessing-5d586b99bb-vq7rp ERROR [io.quarkus.runtime.Application:-1] (Quarkus Main Thread) Failed to start application (with profile [dev]): java.lang.IllegalArgumentException: Unsupported scheme for the http connector in URL: services.v1/event-listener
at io.quarkus.reactivemessaging.http.runtime.HttpSink.<init>(HttpSink.java:64)
at io.quarkus.reactivemessaging.http.runtime.QuarkusHttpConnector.getSubscriberBuilder(QuarkusHttpConnector.java:119)
at io.quarkus.reactivemessaging.http.runtime.QuarkusHttpConnector_Subclass.getSubscriberBuilder$$superforward1(Unknown Source)
at io.quarkus.reactivemessaging.http.runtime.QuarkusHttpConnector_Subclass$$function$$3.apply(Unknown Source)
at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:54)
at io.quarkus.smallrye.reactivemessaging.runtime.devmode.DevModeSupportConnectorFactoryInterceptor.intercept(DevModeSupportConnectorFactoryInterceptor.java:67)
at io.quarkus.smallrye.reactivemessaging.runtime.devmode.DevModeSupportConnectorFactoryInterceptor_Bean.intercept(Unknown Source)
at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:50)
at io.quarkus.smallrye.reactivemessaging.runtime.DuplicatedContextConnectorFactoryInterceptor.intercept(DuplicatedContextConnectorFactoryInterceptor.java:37)
at io.quarkus.smallrye.reactivemessaging.runtime.DuplicatedContextConnectorFactoryInterceptor_Bean.intercept(Unknown Source)
at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
at io.quarkus.arc.impl.AroundInvokeInvocationContext.perform(AroundInvokeInvocationContext.java:41)
at io.quarkus.arc.impl.InvocationContexts.performAroundInvoke(InvocationContexts.java:33)
at io.quarkus.reactivemessaging.http.runtime.QuarkusHttpConnector_Subclass.getSubscriberBuilder(Unknown Source)
at io.quarkus.reactivemessaging.http.runtime.QuarkusHttpConnector_ClientProxy.getSubscriberBuilder(Unknown Source)
at io.smallrye.reactive.messaging.providers.impl.ConnectorFactories.lambda$wrap$5(ConnectorFactories.java:84)
at io.smallrye.reactive.messaging.providers.impl.ConfiguredChannelFactory.createSubscriber(ConfiguredChannelFactory.java:191)
at io.smallrye.reactive.messaging.providers.impl.ConfiguredChannelFactory.register(ConfiguredChannelFactory.java:146)
at io.smallrye.reactive.messaging.providers.impl.ConfiguredChannelFactory.initialize(ConfiguredChannelFactory.java:107)
at io.smallrye.reactive.messaging.providers.impl.ConfiguredChannelFactory_ClientProxy.initialize(Unknown Source)
```
I've checked the [configmap](https://github.com/apache/incubator-kie-kogito-serverless-operator/blob/3b5108ca8c352cb3d81fd736f3e116ed21e66a3f/test/testdata/order-processing/04_v1_configmap_properties.yaml#L25) and there's a property added with that value:
```
mp.messaging.outgoing.kogito_outgoing_stream.url = ${kubernetes:services.v1/event-listener}
```
Here is the whole list of properties defined for the test:
```
kogito.events.processdefinitions.enabled = false
kogito.events.processinstances.enabled = false
kogito.events.usertasks.enabled = false
kogito.events.variables.enabled = false
kogito.service.url = http://orderprocessing.sonataflow-operator-system
mp.messaging.outgoing.kogito-job-service-job-request-events.connector = quarkus-http
mp.messaging.outgoing.kogito-job-service-job-request-events.url = http://localhost/v2/jobs/events
mp.messaging.outgoing.kogito_outgoing_stream.url = ${kubernetes:services.v1/event-listener}
org.kie.kogito.addons.knative.eventing.health-enabled = false
quarkus.devservices.enabled = false
quarkus.http.host = 0.0.0.0
quarkus.http.port = 8080
quarkus.kogito.devservices.enabled = false
quarkus.log.level = INFO
```
I'm unfamiliar with the cause of this exception. Does it ring any bells to you?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "tchughesiv (via GitHub)" <gi...@apache.org>.
tchughesiv commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1860474926
@wmedvede @jordigilh `mp.messaging.outgoing.kogito-processinstances-events.url` should only be configured within the workflow properties when DI is enabled in the platform. with this PR it now appears it's being added to the DI/JS service properties as well. this is the result of the property refactoring that's been done.
`mp.messaging.outgoing.kogito-processinstances-events.url` && `mp.messaging.outgoing.kogito-job-service-job-request-events.url` are **client-side** configs and should only be set in the workflow properties when the corresponding service is enabled in the platform.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "ricardozanini (via GitHub)" <gi...@apache.org>.
ricardozanini commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1863212485
e2e fixed here: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/332
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "wmedvede (via GitHub)" <gi...@apache.org>.
wmedvede commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1860318583
Hello Guys, @jordigilh @ricardozanini @jordigilh here goes some test results with the latest commits:
**Jobs service only**
I think this case is working better now after the last commits. Jobs service is deployed and if we add workflows the properties are set, we still have some errors when we create a WF instance, but we can consider out scope of this PR.
See here please: https://github.com/wmedvede/techpreview2/tree/test-platform-services/platforms/jobservice_as_platform_service_postgresql_openshift
**Data index only**
Don't work, see here please:
https://github.com/wmedvede/techpreview2/tree/test-platform-services/platforms/dataindex_as_platform_service_postgresql_openshift
**Jobs Service and Data Index **
Don't work, see here please: https://github.com/wmedvede/techpreview2/tree/test-platform-services/platforms/dataindex_and_jobservice_as_platform_service_postgresql_openshift
I think both cases well work better if we remove the wrong property
mp.messaging.outgoing.kogito-processinstances-events.url =
http://sonataflow-platform-data-index-service.test-platform3-oc/processes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1856442717
@wmedvede these are the properties generated for the job service once I reuse the same code for the app properties handler:
```
mp.messaging.outgoing.kogito-processinstances-events.url = http://sonataflow-platform-data-index-service.default/processes
mp.messaging.outgoing.kogito-job-service-job-request-events.url = http://sonataflow-platform-jobs-service.default/v2/jobs/events
quarkus.smallrye-health.check."org.kie.kogito.jobs.service.messaging.http.health.knative.KSinkInjectionHealthCheck".enabled = false
quarkus.datasource.reactive.url = postgresql://postgres:5432/sonataflow?search_path=jobs-service
quarkus.smallrye-health.check."io.quarkus.kafka.client.health.KafkaHealthCheck".enabled = false
quarkus.http.port = 8080
quarkus.http.host = 0.0.0.0
org.kie.kogito.addons.knative.eventing.health-enabled = false
quarkus.devservices.enabled = false
quarkus.kogito.devservices.enabled = false
mp.messaging.outgoing.kogito-job-service-job-request-events.connector = quarkus-http
kogito.events.processdefinitions.enabled = false
kogito.events.usertasks.enabled = false
kogito.events.variables.enabled = false
```
Can you confirm that all keys for the job service are accounted for?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1858106400
> > > ```
> > > 1. When deploying Jobs Service or Data Index, the `SonataFlowPlatform` controller must set the config properties for each service accordingly for them to work independently.
> > > 2. During the `SonataFlow` controller reconciliation, if the workflow is not `dev` and if there's a `SonataFlowPlatform` with Jobs Service and DI available in the current context, it should have the workflow application properties being updated to add the required properties for the workflow to connect and send events to those services.
> > > ```
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > That's the use case.
> >
> >
> > Should the properties in each service be any different for the service itself or for the deployment triggered by a sonataflow CR instance? In other words: the properties specific for creating the job service are all included in the properties for a sonataflow CR? or a subset of those?
> > Right now I consider them to be the same.
>
> @jordigilh service properties will be different than workflow properties. sure, they may share a few similar configs but there will be configs that only services use... and configs that only the workflows will use. you should be able to reference what was already done w/ DI vs the workflow properties.. and then modify to fit the needs of job service. e.g. the `NewServiceAppPropertyHandler`.
I'm interested in which properties are specific to the service deployment, which ones are shared between workflows and service deployment and which ones are specific to the workflows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "masayag (via GitHub)" <gi...@apache.org>.
masayag commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1416004279
##########
controllers/platform/services.go:
##########
@@ -99,27 +98,9 @@ func createDataIndexDeployment(ctx context.Context, client client.Client, platfo
liveProbe := readyProbe.DeepCopy()
liveProbe.ProbeHandler.HTTPGet.Path = common.QuarkusHealthPathLive
dataDeployContainer := &corev1.Container{
- Image: common.DataIndexImageBase + common.PersistenceTypeEphemeral,
- Env: []corev1.EnvVar{
- {
- Name: "KOGITO_DATA_INDEX_QUARKUS_PROFILE",
- Value: "http-events-support",
- },
- {
- Name: "QUARKUS_HTTP_CORS",
- Value: "true",
- },
- {
- Name: "QUARKUS_HTTP_CORS_ORIGINS",
- Value: "/.*/",
- },
- },
- Resources: corev1.ResourceRequirements{
- Limits: corev1.ResourceList{
- corev1.ResourceCPU: resource.MustParse("100m"),
- corev1.ResourceMemory: resource.MustParse("256Mi"),
- },
- },
+ Image: svc.GetServiceImageName(common.PersistenceTypeEphemeral),
+ Env: svc.GetEnvironmentVariables(),
+ Resources: svc.GetPodResourceRequirements(),
Review Comment:
per golang convention, this can be simply `ps` (shorten for PlatformService)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "tchughesiv (via GitHub)" <gi...@apache.org>.
tchughesiv commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1416063458
##########
controllers/platform/services.go:
##########
@@ -58,30 +55,32 @@ func (action *serviceAction) Handle(ctx context.Context, platform *operatorapi.S
return nil, err
}
- if err := createDataIndexComponents(ctx, action.client, platform); err != nil {
- return nil, err
+ if platform.Spec.Services.DataIndex != nil {
Review Comment:
no, this flag only affects "prod" workflows. if it's a dev workflow, this flag won't even be checked by the operator. when a user elects to set `platform.spec.services.dataIndex.enabled` to `true` (which is the default), they are saying they want "prod" workflows to use this data index.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "tchughesiv (via GitHub)" <gi...@apache.org>.
tchughesiv commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1418000071
##########
controllers/platform/services.go:
##########
@@ -58,30 +55,32 @@ func (action *serviceAction) Handle(ctx context.Context, platform *operatorapi.S
return nil, err
}
- if err := createDataIndexComponents(ctx, action.client, platform); err != nil {
- return nil, err
+ if platform.Spec.Services.DataIndex != nil {
Review Comment:
@ricardozanini it's worth asking the question though, to @masayag's point... is this a use-case we should be supporting anyway? What IS the use-case? When would an admin or dev need/want to "disable" these services? FWIW, the user could just as easily set the service replicas to 0 themselves. Is this "enabled" field necessary?
I'm good either way... just asking the Q.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "ricardozanini (via GitHub)" <gi...@apache.org>.
ricardozanini commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1424395148
##########
controllers/platform/services/properties.go:
##########
@@ -0,0 +1,98 @@
+// Copyright 2023 Apache Software Foundation (ASF)
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package services
+
+import (
+ "fmt"
+ "strings"
+
+ operatorapi "github.com/apache/incubator-kie-kogito-serverless-operator/api/v1alpha08"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles/common/constants"
+
+ "github.com/magiconair/properties"
+)
+
+func generateReactiveURL(postgresSpec *operatorapi.PersistencePostgreSql, schema string, namespace string, dbName string, port int) string {
+ if len(postgresSpec.JdbcUrl) > 0 && strings.Contains(postgresSpec.JdbcUrl, "currentSchema=") {
+ return strings.Replace(strings.TrimPrefix(postgresSpec.JdbcUrl, "jdbc:"), "currentSchema=", "search_path=", -1)
+ }
+ databaseSchema := schema
+ if len(postgresSpec.ServiceRef.DatabaseSchema) > 0 {
+ databaseSchema = postgresSpec.ServiceRef.DatabaseSchema
+ }
+ databaseNamespace := namespace
+ if len(postgresSpec.ServiceRef.Namespace) > 0 {
+ databaseNamespace = postgresSpec.ServiceRef.Namespace
+ }
+ dataSourcePort := port
+ if postgresSpec.ServiceRef.Port != nil {
+ dataSourcePort = *postgresSpec.ServiceRef.Port
+ }
+ databaseName := dbName
+ if len(postgresSpec.ServiceRef.DatabaseName) > 0 {
+ databaseName = postgresSpec.ServiceRef.DatabaseName
+ }
+ return fmt.Sprintf("%s://%s:%d/%s?search_path=%s", constants.PersistenceTypePostgreSQL, postgresSpec.ServiceRef.Name+"."+databaseNamespace, dataSourcePort, databaseName, databaseSchema)
+}
+
+// GenerateDataIndexApplicationProperties returns the application properties required for the Data Index Service to work when deployed in a production profile
+// and data index' service's spec field `Enabled` set to true
+func GenerateDataIndexApplicationProperties(workflow *operatorapi.SonataFlow, platform *operatorapi.SonataFlowPlatform) *properties.Properties {
+ props := properties.NewProperties()
+ if profiles.IsProdProfile(workflow) && dataIndexEnabled(platform) {
+ di := NewDataIndexService(platform)
+ props.Set(
+ constants.DataIndexServiceURLProperty,
+ fmt.Sprintf("%s://%s.%s/processes", constants.DataIndexServiceURLProtocol, di.GetServiceName(), platform.Namespace),
+ )
+ }
+ return props
+}
+
+// GenerateJobServiceApplicationProperties returns the application properties required for the Job Service to work in a production profile and job service's
+// service's spec field `Enabled` set to true
+func GenerateJobServiceApplicationProperties(workflow *operatorapi.SonataFlow, platform *operatorapi.SonataFlowPlatform) *properties.Properties {
+ props := properties.NewProperties()
+ if profiles.IsProdProfile(workflow) && jobServiceEnabled(platform) {
Review Comment:
Yes, that should work independently.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "wmedvede (via GitHub)" <gi...@apache.org>.
wmedvede commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1425378960
##########
controllers/platform/services/properties.go:
##########
@@ -0,0 +1,98 @@
+// Copyright 2023 Apache Software Foundation (ASF)
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package services
+
+import (
+ "fmt"
+ "strings"
+
+ operatorapi "github.com/apache/incubator-kie-kogito-serverless-operator/api/v1alpha08"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles/common/constants"
+
+ "github.com/magiconair/properties"
+)
+
+func generateReactiveURL(postgresSpec *operatorapi.PersistencePostgreSql, schema string, namespace string, dbName string, port int) string {
+ if len(postgresSpec.JdbcUrl) > 0 && strings.Contains(postgresSpec.JdbcUrl, "currentSchema=") {
+ return strings.Replace(strings.TrimPrefix(postgresSpec.JdbcUrl, "jdbc:"), "currentSchema=", "search_path=", -1)
+ }
+ databaseSchema := schema
+ if len(postgresSpec.ServiceRef.DatabaseSchema) > 0 {
+ databaseSchema = postgresSpec.ServiceRef.DatabaseSchema
+ }
+ databaseNamespace := namespace
+ if len(postgresSpec.ServiceRef.Namespace) > 0 {
+ databaseNamespace = postgresSpec.ServiceRef.Namespace
+ }
+ dataSourcePort := port
+ if postgresSpec.ServiceRef.Port != nil {
+ dataSourcePort = *postgresSpec.ServiceRef.Port
+ }
+ databaseName := dbName
+ if len(postgresSpec.ServiceRef.DatabaseName) > 0 {
+ databaseName = postgresSpec.ServiceRef.DatabaseName
+ }
+ return fmt.Sprintf("%s://%s:%d/%s?search_path=%s", constants.PersistenceTypePostgreSQL, postgresSpec.ServiceRef.Name+"."+databaseNamespace, dataSourcePort, databaseName, databaseSchema)
+}
+
+// GenerateDataIndexApplicationProperties returns the application properties required for the Data Index Service to work when deployed in a production profile
+// and data index' service's spec field `Enabled` set to true
+func GenerateDataIndexApplicationProperties(workflow *operatorapi.SonataFlow, platform *operatorapi.SonataFlowPlatform) *properties.Properties {
+ props := properties.NewProperties()
+ if profiles.IsProdProfile(workflow) && dataIndexEnabled(platform) {
+ di := NewDataIndexService(platform)
+ props.Set(
+ constants.DataIndexServiceURLProperty,
+ fmt.Sprintf("%s://%s.%s/processes", constants.DataIndexServiceURLProtocol, di.GetServiceName(), platform.Namespace),
+ )
+ }
+ return props
+}
+
+// GenerateJobServiceApplicationProperties returns the application properties required for the Job Service to work in a production profile and job service's
+// service's spec field `Enabled` set to true
+func GenerateJobServiceApplicationProperties(workflow *operatorapi.SonataFlow, platform *operatorapi.SonataFlowPlatform) *properties.Properties {
+ props := properties.NewProperties()
+ if profiles.IsProdProfile(workflow) && jobServiceEnabled(platform) {
Review Comment:
@tchughesiv This change is because in order for the workflows to work, (when configured) we need the jobs-service and data-index to be deployed first, but also, it might occur, that we push the platform into the namespace, and then, we push the workflows. So, what we are looking for with this change is that the deployment of the services are not dependent on any particular workflow.
That's basically the aim of this change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1855050916
Let me know if you want me to squash the commits before merging.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1424841769
##########
controllers/profiles/common/properties/application.go:
##########
@@ -27,46 +27,30 @@ import (
"strings"
"github.com/apache/incubator-kie-kogito-serverless-operator/controllers/discovery"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/platform/services"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles/common/constants"
"github.com/magiconair/properties"
"k8s.io/klog/v2"
operatorapi "github.com/apache/incubator-kie-kogito-serverless-operator/api/v1alpha08"
- "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles"
"github.com/apache/incubator-kie-kogito-serverless-operator/log"
)
-const (
- ConfigMapWorkflowPropsVolumeName = "workflow-properties"
- kogitoServiceUrlProperty = "kogito.service.url"
- kogitoServiceUrlProtocol = "http"
- dataIndexServiceUrlProperty = "mp.messaging.outgoing.kogito-processinstances-events.url"
- kafkaSmallRyeHealthProperty = "quarkus.smallrye-health.check.\"io.quarkus.kafka.client.health.KafkaHealthCheck\".enabled"
- dataIndexServiceUrlProtocol = "http"
-
- DataIndexImageBase = "quay.io/kiegroup/kogito-data-index-"
- DataIndexName = "data-index-service"
-
- PersistenceTypeEphemeral = "ephemeral"
- PersistenceTypePostgressql = "postgresql"
-
- microprofileServiceCatalogPropertyPrefix = "org.kie.kogito.addons.discovery."
- discoveryLikePropertyPattern = "^\\${(kubernetes|knative|openshift):(.*)}$"
+var (
+ immutableApplicationProperties = fmt.Sprintf("quarkus.http.port=%d\n"+
+ "quarkus.http.host=0.0.0.0\n"+
+ // We disable the Knative health checks to not block the dev pod to run if Knative objects are not available
+ // See: https://kiegroup.github.io/kogito-docs/serverlessworkflow/latest/eventing/consume-produce-events-with-knative-eventing.html#ref-knative-eventing-add-on-source-configuration
+ "org.kie.kogito.addons.knative.eventing.health-enabled=false\n"+
+ "quarkus.devservices.enabled=false\n"+
+ "quarkus.kogito.devservices.enabled=false\n", constants.DefaultHTTPWorkflowPortInt)
+
+ discoveryLikePropertyExpr = regexp.MustCompile(constants.DiscoveryLikePropertyPattern)
Review Comment:
Done, moved constant to this file and made it private
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "ricardozanini (via GitHub)" <gi...@apache.org>.
ricardozanini commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1858106123
Hi @jordigilh
> in other words: the properties specific for creating the job service are all included in the properties for a sonataflow CR? or a subset of those?
> Right now I consider them to be the same.
No, they are not the same. I think it was there that things messed a little bit. Take a look at this repo we shared at the beginning of the development:
https://github.com/flows-examples/techpreview2/tree/main/infra/dataindex_and_jobservice
You can see the JS Deployment and the `application.properties` for it there. That's exactly what we were expecting for this PR to do.
And then, if you see the use case:
https://github.com/flows-examples/techpreview2/tree/main?tab=readme-ov-file#use-case-3
It's explained there how it is supposed to be working. The workflow has yet another set of properties:
https://github.com/flows-examples/techpreview2/blob/main/workflows/sonataflow-callbackstatetimeouts-persistence/02-configmap_callbackstatetimeouts-props
That's why I asked to separate the props in the first place last week between the services and the workflow. I thought it was clear since these use cases were already working and defined in that repo. I'm sorry if I made an incorrect assumption.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "wmedvede (via GitHub)" <gi...@apache.org>.
wmedvede commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1858302685
@jordigilh
**Properties for the workflows:**
kogito.service.url (we always set, it's the cluster dns url to my self, the operator was already doing it for the workflows)
**This two properties are needed to connect with the jobs service and we **must always set**** (no matter if jobs service is present or not)
mp.messaging.outgoing.kogito-job-service-job-request-events.connector=quarkus-http
mp.messaging.outgoing.kogito-job-service-job-request-events.url= http://jobs-service-postgresql/v2/jobs/events
If the jobs service is **NOT installed**, we still set the url, and we invent it :), to http://jobs-service/v2/jobs/events, np, we can manage in another PR to fine tune it. And, if the jobs service is installed we set the correct cluster dns url)
**Properties to connect with the data-index:**
**If data index is not present we set this:** (all in false)
kogito.events.processinstances.enabled=false
kogito.events.processdefinitions.enabled=false
kogito.events.usertasks.enabled=false
kogito.events.variables.enabled=false
**If data index is present then we set this instead:**
kogito.events.processinstances.enabled=true **(only this in true)**
kogito.events.processdefinitions.enabled=false
kogito.events.usertasks.enabled=false
kogito.events.variables.enabled=false
**And, since data index is present we add this one with the correct data index cluster dns**
mp.messaging.outgoing.kogito-processinstances-events.url=http://data-index-service-postgresql/processes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "wmedvede (via GitHub)" <gi...@apache.org>.
wmedvede commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1858306411
> quarkus.datasource.jdbc.url
@jordigilh Your observation is correct, the quarkus.datasource.jdbc.url we are already setting in the env vars, **so we don't need to** set as property, It was added there for the example. But yeah, your comment is correct, well noted! , see that things like the database user and password we also set as env vars.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "ricardozanini (via GitHub)" <gi...@apache.org>.
ricardozanini commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1415440374
##########
controllers/platform/services.go:
##########
@@ -99,27 +98,9 @@ func createDataIndexDeployment(ctx context.Context, client client.Client, platfo
liveProbe := readyProbe.DeepCopy()
liveProbe.ProbeHandler.HTTPGet.Path = common.QuarkusHealthPathLive
dataDeployContainer := &corev1.Container{
- Image: common.DataIndexImageBase + common.PersistenceTypeEphemeral,
- Env: []corev1.EnvVar{
- {
- Name: "KOGITO_DATA_INDEX_QUARKUS_PROFILE",
- Value: "http-events-support",
- },
- {
- Name: "QUARKUS_HTTP_CORS",
- Value: "true",
- },
- {
- Name: "QUARKUS_HTTP_CORS_ORIGINS",
- Value: "/.*/",
- },
- },
- Resources: corev1.ResourceRequirements{
- Limits: corev1.ResourceList{
- corev1.ResourceCPU: resource.MustParse("100m"),
- corev1.ResourceMemory: resource.MustParse("256Mi"),
- },
- },
+ Image: svc.GetServiceImageName(common.PersistenceTypeEphemeral),
+ Env: svc.GetEnvironmentVariables(),
+ Resources: svc.GetPodResourceRequirements(),
Review Comment:
`svc` is an alias for Kubernetes Services, can we change this since this context is different, please?
##########
controllers/profiles/common/app_properties.go:
##########
@@ -38,34 +43,313 @@ import (
)
const (
- ConfigMapWorkflowPropsVolumeName = "workflow-properties"
- kogitoServiceUrlProperty = "kogito.service.url"
- kogitoServiceUrlProtocol = "http"
- dataIndexServiceUrlProperty = "mp.messaging.outgoing.kogito-processinstances-events.url"
- kafkaSmallRyeHealthProperty = "quarkus.smallrye-health.check.\"io.quarkus.kafka.client.health.KafkaHealthCheck\".enabled"
- dataIndexServiceUrlProtocol = "http"
-
- DataIndexImageBase = "quay.io/kiegroup/kogito-data-index-"
- DataIndexName = "data-index-service"
-
- PersistenceTypeEphemeral = "ephemeral"
- PersistenceTypePostgressql = "postgresql"
-
+ ConfigMapWorkflowPropsVolumeName = "workflow-properties"
+ kogitoServiceUrlProperty = "kogito.service.url"
+ kogitoServiceUrlProtocol = "http"
+ dataIndexServiceUrlProperty = "mp.messaging.outgoing.kogito-processinstances-events.url"
+ kafkaSmallRyeHealthProperty = "quarkus.smallrye-health.check.\"io.quarkus.kafka.client.health.KafkaHealthCheck\".enabled"
+ dataIndexServiceUrlProtocol = "http"
+ jobServiceURLProperty = "mp.messaging.outgoing.kogito-job-service-job-request-events.url"
+ jobServiceKafkaSinkInjectionHealthCheck = `quarkus.smallrye-health.check."org.kie.kogito.jobs.service.messaging.http.health.knative.KSinkInjectionHealthCheck".enabled`
+ jobServiceStatusChangeEventsProperty = "kogito.jobs-service.http.job-status-change-events"
+ jobServiceStatusChangeEventsURL = "mp.messaging.outgoing.kogito-job-service-job-status-events-http.url"
+ jobServiceURLProtocol = "http"
+ jobServiceDataSourceReactiveURLProperty = "quarkus.datasource.reactive.url"
+
+ imageNamePrefix = "quay.io/kiegroup/kogito"
+ DataIndexServiceName = "data-index-service"
+ DataIndexName = "data-index"
+ JobServiceName = "jobs-service"
+ PersistenceTypeEphemeral = "ephemeral"
+ PersistenceTypePostgressql = "postgresql"
microprofileServiceCatalogPropertyPrefix = "org.kie.kogito.addons.discovery."
discoveryLikePropertyPattern = "^\\${(kubernetes|knative|openshift):(.*)}$"
+
+ defaultDatabaseName string = "sonataflow"
+ defaultPostgreSQLPort int = 5432
+)
+
+var (
+ immutableApplicationProperties = "quarkus.http.port=" + DefaultHTTPWorkflowPortIntStr.String() + "\n" +
+ "quarkus.http.host=0.0.0.0\n" +
+ // We disable the Knative health checks to not block the dev pod to run if Knative objects are not available
+ // See: https://kiegroup.github.io/kogito-docs/serverlessworkflow/latest/eventing/consume-produce-events-with-knative-eventing.html#ref-knative-eventing-add-on-source-configuration
+ "org.kie.kogito.addons.knative.eventing.health-enabled=false\n" +
+ "quarkus.devservices.enabled=false\n" +
+ "quarkus.kogito.devservices.enabled=false\n"
+
+ discoveryLikePropertyExpr = regexp.MustCompile(discoveryLikePropertyPattern)
+ _ AppPropertyHandler = &appPropertyHandler{}
)
-var immutableApplicationProperties = "quarkus.http.port=" + DefaultHTTPWorkflowPortIntStr.String() + "\n" +
- "quarkus.http.host=0.0.0.0\n" +
- // We disable the Knative health checks to not block the dev pod to run if Knative objects are not available
- // See: https://kiegroup.github.io/kogito-docs/serverlessworkflow/latest/eventing/consume-produce-events-with-knative-eventing.html#ref-knative-eventing-add-on-source-configuration
- "org.kie.kogito.addons.knative.eventing.health-enabled=false\n" +
- "quarkus.devservices.enabled=false\n" +
- "quarkus.kogito.devservices.enabled=false\n"
+type PlatformService interface {
Review Comment:
We are mixing Application properties with object creations in this file. This `PlatformService` must be within the package `platform` and not be exported.
##########
controllers/profiles/common/app_properties.go:
##########
@@ -38,34 +43,313 @@ import (
)
const (
- ConfigMapWorkflowPropsVolumeName = "workflow-properties"
- kogitoServiceUrlProperty = "kogito.service.url"
- kogitoServiceUrlProtocol = "http"
- dataIndexServiceUrlProperty = "mp.messaging.outgoing.kogito-processinstances-events.url"
- kafkaSmallRyeHealthProperty = "quarkus.smallrye-health.check.\"io.quarkus.kafka.client.health.KafkaHealthCheck\".enabled"
- dataIndexServiceUrlProtocol = "http"
-
- DataIndexImageBase = "quay.io/kiegroup/kogito-data-index-"
- DataIndexName = "data-index-service"
-
- PersistenceTypeEphemeral = "ephemeral"
- PersistenceTypePostgressql = "postgresql"
-
+ ConfigMapWorkflowPropsVolumeName = "workflow-properties"
+ kogitoServiceUrlProperty = "kogito.service.url"
+ kogitoServiceUrlProtocol = "http"
+ dataIndexServiceUrlProperty = "mp.messaging.outgoing.kogito-processinstances-events.url"
+ kafkaSmallRyeHealthProperty = "quarkus.smallrye-health.check.\"io.quarkus.kafka.client.health.KafkaHealthCheck\".enabled"
+ dataIndexServiceUrlProtocol = "http"
+ jobServiceURLProperty = "mp.messaging.outgoing.kogito-job-service-job-request-events.url"
+ jobServiceKafkaSinkInjectionHealthCheck = `quarkus.smallrye-health.check."org.kie.kogito.jobs.service.messaging.http.health.knative.KSinkInjectionHealthCheck".enabled`
+ jobServiceStatusChangeEventsProperty = "kogito.jobs-service.http.job-status-change-events"
+ jobServiceStatusChangeEventsURL = "mp.messaging.outgoing.kogito-job-service-job-status-events-http.url"
+ jobServiceURLProtocol = "http"
+ jobServiceDataSourceReactiveURLProperty = "quarkus.datasource.reactive.url"
+
+ imageNamePrefix = "quay.io/kiegroup/kogito"
+ DataIndexServiceName = "data-index-service"
+ DataIndexName = "data-index"
+ JobServiceName = "jobs-service"
+ PersistenceTypeEphemeral = "ephemeral"
+ PersistenceTypePostgressql = "postgresql"
microprofileServiceCatalogPropertyPrefix = "org.kie.kogito.addons.discovery."
discoveryLikePropertyPattern = "^\\${(kubernetes|knative|openshift):(.*)}$"
+
+ defaultDatabaseName string = "sonataflow"
+ defaultPostgreSQLPort int = 5432
+)
Review Comment:
There are too many properties here that belong to services. Can we please have an app properties handler for workflows and other for the services?
`platform` -> handler for Services
`profiles` -> handler for workflows
We might have a common module that defines a strategy for handling properties, but each module should have its own responsibility accordingly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "ricardozanini (via GitHub)" <gi...@apache.org>.
ricardozanini commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1419404469
##########
controllers/platform/services.go:
##########
@@ -58,30 +55,32 @@ func (action *serviceAction) Handle(ctx context.Context, platform *operatorapi.S
return nil, err
}
- if err := createDataIndexComponents(ctx, action.client, platform); err != nil {
- return nil, err
+ if platform.Spec.Services.DataIndex != nil {
Review Comment:
I keep asking these to myself. Let's reevaluate the need for it. I remember we discussed how to have the platform deploy the services and that's what we came up with. If we enable deployment, we should also disable it. We must have a response to irt or change the structure in a way that "disable" won't make sense.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1847210408
> @jordigilh does your PR descrip still match the changes you're making here? if not, could you update it? thanks!
Update.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "masayag (via GitHub)" <gi...@apache.org>.
masayag commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1416056317
##########
controllers/platform/services.go:
##########
@@ -99,27 +98,9 @@ func createDataIndexDeployment(ctx context.Context, client client.Client, platfo
liveProbe := readyProbe.DeepCopy()
liveProbe.ProbeHandler.HTTPGet.Path = common.QuarkusHealthPathLive
dataDeployContainer := &corev1.Container{
Review Comment:
Thank you @tchughesiv :-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1408269468
##########
controllers/profiles/common/app_properties.go:
##########
@@ -45,27 +45,59 @@ const (
kafkaSmallRyeHealthProperty = "quarkus.smallrye-health.check.\"io.quarkus.kafka.client.health.KafkaHealthCheck\".enabled"
dataIndexServiceUrlProtocol = "http"
- DataIndexImageBase = "quay.io/kiegroup/kogito-data-index-"
- DataIndexName = "data-index-service"
-
- PersistenceTypeEphemeral = "ephemeral"
- PersistenceTypePostgressql = "postgresql"
-
+ imageNamePrefix = "quay.io/kiegroup/kogito"
+ DataIndexServiceName = "data-index-service"
+ DataIndexName = "data-index"
+ JobServiceName = "jobs-service"
+ ImageTag = "1.44"
+ PersistenceTypeEphemeral = "ephemeral"
+ PersistenceTypePostgressql = "postgresql"
microprofileServiceCatalogPropertyPrefix = "org.kie.kogito.addons.discovery."
discoveryLikePropertyPattern = "^\\${(kubernetes|knative|openshift):(.*)}$"
)
-var immutableApplicationProperties = "quarkus.http.port=" + DefaultHTTPWorkflowPortIntStr.String() + "\n" +
- "quarkus.http.host=0.0.0.0\n" +
- // We disable the Knative health checks to not block the dev pod to run if Knative objects are not available
- // See: https://kiegroup.github.io/kogito-docs/serverlessworkflow/latest/eventing/consume-produce-events-with-knative-eventing.html#ref-knative-eventing-add-on-source-configuration
- "org.kie.kogito.addons.knative.eventing.health-enabled=false\n" +
- "quarkus.devservices.enabled=false\n" +
- "quarkus.kogito.devservices.enabled=false\n"
+var (
+ immutableApplicationProperties = "quarkus.http.port=" + DefaultHTTPWorkflowPortIntStr.String() + "\n" +
+ "quarkus.http.host=0.0.0.0\n" +
+ // We disable the Knative health checks to not block the dev pod to run if Knative objects are not available
+ // See: https://kiegroup.github.io/kogito-docs/serverlessworkflow/latest/eventing/consume-produce-events-with-knative-eventing.html#ref-knative-eventing-add-on-source-configuration
+ "org.kie.kogito.addons.knative.eventing.health-enabled=false\n" +
+ "quarkus.devservices.enabled=false\n" +
+ "quarkus.kogito.devservices.enabled=false\n"
+
+ discoveryLikePropertyExpr = regexp.MustCompile(discoveryLikePropertyPattern)
+ _ AppPropertyHandler = &appPropertyHandler{}
+)
-var discoveryLikePropertyExpr = regexp.MustCompile(discoveryLikePropertyPattern)
+const (
+ DataIndexService = iota
+ JobService
+)
-var _ AppPropertyHandler = &appPropertyHandler{}
+// GetContainerName returns the name of the service's container in the deployment.
+func GetContainerName(stype int) string {
Review Comment:
It's used in the platform package, so it needs to be public https://github.com/apache/incubator-kie-kogito-serverless-operator/blob/cba7063ca24d0350252b41e1de0f3ea0dfe2b0b8/controllers/platform/services.go#L186
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "tchughesiv (via GitHub)" <gi...@apache.org>.
tchughesiv commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1408670546
##########
controllers/profiles/common/app_properties_test.go:
##########
@@ -128,7 +128,7 @@ func Test_appPropertyHandler_WithServicesWithUserOverrides(t *testing.T) {
generatedProps, propsErr = properties.LoadString(props)
assert.NoError(t, propsErr)
assert.Equal(t, 9, len(generatedProps.Keys()))
- assert.Equal(t, "http://"+platform.Name+"-"+DataIndexName+"."+platform.Namespace+"/processes", generatedProps.GetString(dataIndexServiceUrlProperty, ""))
+ assert.Equal(t, "http://"+platform.Name+"-"+DataIndexServiceName+"."+platform.Namespace+"/processes", generatedProps.GetString(dataIndexServiceUrlProperty, ""))
Review Comment:
missed this earlier... please add a similar unit test that ensures job service is getting added as an app property
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1408285085
##########
controllers/profiles/common/app_properties.go:
##########
@@ -45,27 +45,59 @@ const (
kafkaSmallRyeHealthProperty = "quarkus.smallrye-health.check.\"io.quarkus.kafka.client.health.KafkaHealthCheck\".enabled"
dataIndexServiceUrlProtocol = "http"
- DataIndexImageBase = "quay.io/kiegroup/kogito-data-index-"
- DataIndexName = "data-index-service"
-
- PersistenceTypeEphemeral = "ephemeral"
- PersistenceTypePostgressql = "postgresql"
-
+ imageNamePrefix = "quay.io/kiegroup/kogito"
+ DataIndexServiceName = "data-index-service"
+ DataIndexName = "data-index"
+ JobServiceName = "jobs-service"
+ ImageTag = "1.44"
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "masayag (via GitHub)" <gi...@apache.org>.
masayag commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1416010012
##########
controllers/platform/services.go:
##########
@@ -99,27 +98,9 @@ func createDataIndexDeployment(ctx context.Context, client client.Client, platfo
liveProbe := readyProbe.DeepCopy()
liveProbe.ProbeHandler.HTTPGet.Path = common.QuarkusHealthPathLive
dataDeployContainer := &corev1.Container{
Review Comment:
@ricardozanini is the `data` prefix of `dataDeployContainer` related to the data-index from the origin function purpose or serves a different meaning here? If it ties to `data-index`, can we omit it from the name of this variable to make this function more generic?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "ricardozanini (via GitHub)" <gi...@apache.org>.
ricardozanini commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1416040099
##########
controllers/platform/services.go:
##########
@@ -99,27 +98,9 @@ func createDataIndexDeployment(ctx context.Context, client client.Client, platfo
liveProbe := readyProbe.DeepCopy()
liveProbe.ProbeHandler.HTTPGet.Path = common.QuarkusHealthPathLive
dataDeployContainer := &corev1.Container{
Review Comment:
+1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "masayag (via GitHub)" <gi...@apache.org>.
masayag commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1415999615
##########
controllers/platform/services.go:
##########
@@ -58,30 +55,32 @@ func (action *serviceAction) Handle(ctx context.Context, platform *operatorapi.S
return nil, err
}
- if err := createDataIndexComponents(ctx, action.client, platform); err != nil {
- return nil, err
+ if platform.Spec.Services.DataIndex != nil {
+ if err := createServiceComponents(ctx, action.client, platform, common.NewDataIndexService(platform)); err != nil {
+ return nil, err
+ }
+ }
+
+ if platform.Spec.Services.JobService != nil {
Review Comment:
same here, Should we get into this block if `platform.Spec.Services.JobService.Enabled == false`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "masayag (via GitHub)" <gi...@apache.org>.
masayag commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1416010012
##########
controllers/platform/services.go:
##########
@@ -99,27 +98,9 @@ func createDataIndexDeployment(ctx context.Context, client client.Client, platfo
liveProbe := readyProbe.DeepCopy()
liveProbe.ProbeHandler.HTTPGet.Path = common.QuarkusHealthPathLive
dataDeployContainer := &corev1.Container{
Review Comment:
@ricardozanini is the `data` prefix related to the data-index from the origin function purpose or serves a different meaning here? If it ties to `data-index`, can we omit it from the name of this variable to make this function more generic?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "ricardozanini (via GitHub)" <gi...@apache.org>.
ricardozanini commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1422774946
##########
controllers/platform/services.go:
##########
@@ -58,30 +55,32 @@ func (action *serviceAction) Handle(ctx context.Context, platform *operatorapi.S
return nil, err
}
- if err := createDataIndexComponents(ctx, action.client, platform); err != nil {
- return nil, err
+ if platform.Spec.Services.DataIndex != nil {
Review Comment:
Won't we replicate the spec in the cluster scope? I'd rather have the change once and propagate to the cluster-scoped requirements.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "tchughesiv (via GitHub)" <gi...@apache.org>.
tchughesiv commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1422748685
##########
controllers/platform/services.go:
##########
@@ -58,30 +55,32 @@ func (action *serviceAction) Handle(ctx context.Context, platform *operatorapi.S
return nil, err
}
- if err := createDataIndexComponents(ctx, action.client, platform); err != nil {
- return nil, err
+ if platform.Spec.Services.DataIndex != nil {
Review Comment:
I also think this all sounds reasonable, with `RollingUpdate` as the default when this new "flag" is not set. @ricardozanini I'm not sure the cluster-scoped changes would need to wait for this though... what what your thinking with that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1424400250
##########
controllers/platform/services/properties.go:
##########
@@ -0,0 +1,98 @@
+// Copyright 2023 Apache Software Foundation (ASF)
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package services
+
+import (
+ "fmt"
+ "strings"
+
+ operatorapi "github.com/apache/incubator-kie-kogito-serverless-operator/api/v1alpha08"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles/common/constants"
+
+ "github.com/magiconair/properties"
+)
+
+func generateReactiveURL(postgresSpec *operatorapi.PersistencePostgreSql, schema string, namespace string, dbName string, port int) string {
+ if len(postgresSpec.JdbcUrl) > 0 && strings.Contains(postgresSpec.JdbcUrl, "currentSchema=") {
+ return strings.Replace(strings.TrimPrefix(postgresSpec.JdbcUrl, "jdbc:"), "currentSchema=", "search_path=", -1)
Review Comment:
so `currentSchema` is optional then? Shall I just replace it when found, otherwise append `[?]search_path=<schema>`?
```
postgresql://postgres:5432/sonataflow?search_path=<schema>
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1424403721
##########
controllers/platform/services/services.go:
##########
@@ -0,0 +1,298 @@
+// Copyright 2023 Red Hat, Inc. and/or its affiliates
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package services
+
+import (
+ "fmt"
+ "strconv"
+
+ corev1 "k8s.io/api/core/v1"
+ "k8s.io/apimachinery/pkg/api/resource"
+
+ operatorapi "github.com/apache/incubator-kie-kogito-serverless-operator/api/v1alpha08"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles/common/constants"
+
+ "github.com/apache/incubator-kie-kogito-serverless-operator/version"
+ "github.com/imdario/mergo"
+)
+
+type Platform interface {
+ // GetContainerName returns the name of the service's container in the deployment.
+ GetContainerName() string
+ // GetServiceImageName returns the image name of the service's container. It takes in the service and persistence types and returns a string
+ // that contains the FQDN of the image, including the tag.
+ GetServiceImageName(persistenceName string) string
+ // GetServiceName returns the name of the kubernetes service prefixed with the platform name
+ GetServiceName() string
+ // GetServiceCmName returns the name of the configmap associated to the service
+ GetServiceCmName() string
+ // GetEnvironmentVariables returns the env variables to be injected to the service container
+ GetEnvironmentVariables() []corev1.EnvVar
+ // GetResourceLimits returns the pod's memory and CPU resource requirements
+ // Values for job service taken from
+ // https://github.com/parodos-dev/orchestrator-helm-chart/blob/52d09eda56fdbed3060782df29847c97f172600f/charts/orchestrator/values.yaml#L68-L72
+ GetPodResourceRequirements() corev1.ResourceRequirements
+ // GetReplicaCountForService Returns the default pod replica count for the given service
+ GetReplicaCount() int32
Review Comment:
ok so it should always set to 1 regardless of what it's defined elsewhere, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1424402672
##########
controllers/profiles/common/properties/application.go:
##########
@@ -27,46 +27,30 @@ import (
"strings"
"github.com/apache/incubator-kie-kogito-serverless-operator/controllers/discovery"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/platform/services"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles/common/constants"
"github.com/magiconair/properties"
"k8s.io/klog/v2"
operatorapi "github.com/apache/incubator-kie-kogito-serverless-operator/api/v1alpha08"
- "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles"
"github.com/apache/incubator-kie-kogito-serverless-operator/log"
)
-const (
- ConfigMapWorkflowPropsVolumeName = "workflow-properties"
- kogitoServiceUrlProperty = "kogito.service.url"
- kogitoServiceUrlProtocol = "http"
- dataIndexServiceUrlProperty = "mp.messaging.outgoing.kogito-processinstances-events.url"
- kafkaSmallRyeHealthProperty = "quarkus.smallrye-health.check.\"io.quarkus.kafka.client.health.KafkaHealthCheck\".enabled"
- dataIndexServiceUrlProtocol = "http"
-
- DataIndexImageBase = "quay.io/kiegroup/kogito-data-index-"
- DataIndexName = "data-index-service"
-
- PersistenceTypeEphemeral = "ephemeral"
- PersistenceTypePostgressql = "postgresql"
-
- microprofileServiceCatalogPropertyPrefix = "org.kie.kogito.addons.discovery."
- discoveryLikePropertyPattern = "^\\${(kubernetes|knative|openshift):(.*)}$"
+var (
+ immutableApplicationProperties = fmt.Sprintf("quarkus.http.port=%d\n"+
+ "quarkus.http.host=0.0.0.0\n"+
+ // We disable the Knative health checks to not block the dev pod to run if Knative objects are not available
+ // See: https://kiegroup.github.io/kogito-docs/serverlessworkflow/latest/eventing/consume-produce-events-with-knative-eventing.html#ref-knative-eventing-add-on-source-configuration
+ "org.kie.kogito.addons.knative.eventing.health-enabled=false\n"+
+ "quarkus.devservices.enabled=false\n"+
+ "quarkus.kogito.devservices.enabled=false\n", constants.DefaultHTTPWorkflowPortInt)
+
+ discoveryLikePropertyExpr = regexp.MustCompile(constants.DiscoveryLikePropertyPattern)
Review Comment:
ack
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1857125952
> 1. When deploying Jobs Service or Data Index, the `SonataFlowPlatform` controller must set the config properties for each service accordingly for them to work independently.
> 2. During the `SonataFlow` controller reconciliation, if the workflow is not `dev` and if there's a `SonataFlowPlatform` with Jobs Service and DI available in the current context, it should have the workflow application properties being updated to add the required properties for the workflow to connect and send events to those services.
>
>
> That's the use case.
Should the properties in each service be any different for the service itself or for the deployment triggered by a sonataflow CR instance? In other words: the properties specific for creating the job service are all included in the properties for a sonataflow CR? or a subset of those?
Right now I consider them to be the same.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "tchughesiv (via GitHub)" <gi...@apache.org>.
tchughesiv commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1408672193
##########
controllers/profiles/common/app_properties.go:
##########
@@ -156,7 +194,7 @@ func (a *appPropertyHandler) withDataIndexServiceUrl() AppPropertyHandler {
if profiles.IsProdProfile(a.workflow) && dataIndexEnabled(a.platform) {
a.addDefaultMutableProperty(
dataIndexServiceUrlProperty,
- fmt.Sprintf("%s://%s.%s/processes", dataIndexServiceUrlProtocol, GetDataIndexName(a.platform), a.platform.Namespace),
+ fmt.Sprintf("%s://%s.%s/processes", dataIndexServiceUrlProtocol, GetServiceName(a.platform.Name, DataIndexService), a.platform.Namespace),
Review Comment:
does a job service app property also need to be added here so that workflows know how to communicate to it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "masayag (via GitHub)" <gi...@apache.org>.
masayag commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1415999615
##########
controllers/platform/services.go:
##########
@@ -58,30 +55,32 @@ func (action *serviceAction) Handle(ctx context.Context, platform *operatorapi.S
return nil, err
}
- if err := createDataIndexComponents(ctx, action.client, platform); err != nil {
- return nil, err
+ if platform.Spec.Services.DataIndex != nil {
+ if err := createServiceComponents(ctx, action.client, platform, common.NewDataIndexService(platform)); err != nil {
+ return nil, err
+ }
+ }
+
+ if platform.Spec.Services.JobService != nil {
Review Comment:
same here, should we get into this block if `platform.Spec.Services.JobService.Enabled == false`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "tchughesiv (via GitHub)" <gi...@apache.org>.
tchughesiv commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1416033099
##########
controllers/platform/services.go:
##########
@@ -58,30 +55,32 @@ func (action *serviceAction) Handle(ctx context.Context, platform *operatorapi.S
return nil, err
}
- if err := createDataIndexComponents(ctx, action.client, platform); err != nil {
- return nil, err
+ if platform.Spec.Services.DataIndex != nil {
Review Comment:
it's correct as it is. the `Enabled` field does not control whether a service is deployed. instead, it controls whether its used by the "prod" workflows -
https://github.com/apache/incubator-kie-kogito-serverless-operator/blob/a96add5c995496ef2d02f313114efed7d74a4725/api/v1alpha08/sonataflowplatform_services_types.go#L27-L29
##########
controllers/platform/services.go:
##########
@@ -58,30 +55,32 @@ func (action *serviceAction) Handle(ctx context.Context, platform *operatorapi.S
return nil, err
}
- if err := createDataIndexComponents(ctx, action.client, platform); err != nil {
- return nil, err
+ if platform.Spec.Services.DataIndex != nil {
+ if err := createServiceComponents(ctx, action.client, platform, common.NewDataIndexService(platform)); err != nil {
+ return nil, err
+ }
+ }
+
+ if platform.Spec.Services.JobService != nil {
Review Comment:
same
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "wmedvede (via GitHub)" <gi...@apache.org>.
wmedvede commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1855541871
Hello Guys @jordigilh @tchughesiv @ricardozanini , I'll add some test results here. (the last commits added by @jordigilh were used)
**Note:**
In general I have added reproducers for the tests cases here: https://github.com/wmedvede/techpreview2/tree/test-platform-services, feel to clone and give it a try locally
**Case 1:**
We deploy the platform with jobs service + postgresql persistence (no workflows):
**reproducer here:** https://github.com/wmedvede/techpreview2/blob/1a2cc37f4f9a9c1423db873da847ee409a576e4d/platforms/jobservice_as_platform_service_postgresql/Readme.txt
What we can see is that the jobs service tries to start, but, no reactive url is created, and thus it fails.
These are the options in my opinion:
**Option 1)** we don't let it start, since we have no workflows yet.
**Option 2)** we let it start, in that case, we need the reactive datasource URL
I think we have discussed already in favor of option 2.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1856395504
> Hello Guys @jordigilh @tchughesiv @ricardozanini , I'll add some test results here. (the last commits added by @jordigilh were used)
>
> **Note:** In general I have added reproducers for the tests cases here: https://github.com/wmedvede/techpreview2/tree/test-platform-services, feel to clone and give it a try locally
>
> **Case 1:** We deploy the platform with jobs service + postgresql persistence (no workflows):
>
> **reproducer here:** https://github.com/wmedvede/techpreview2/blob/1a2cc37f4f9a9c1423db873da847ee409a576e4d/platforms/jobservice_as_platform_service_postgresql/Readme.txt
>
> What we can see is that the jobs service tries to start, but, no reactive url is created, and thus it fails. These are the options in my opinion:
>
> **Option 1)** we don't let it start, since we have no workflows yet.
>
> **Option 2)** we let it start, in that case, we need the reactive datasource URL I think we have discussed already in favor of option 2.
>
> 
>
> These are the generated properties:
>
> 
There are 2 issues I noticed in the logs:
* The controller keeps dumping this error due to a schema validation in the persistence struct for postgreSQL:
https://github.com/apache/incubator-kie-kogito-serverless-operator/blob/3b5108ca8c352cb3d81fd736f3e116ed21e66a3f/api/v1alpha08/sonataflowplatform_services_types.go#L47C4-L48
```
E1214 16:50:41.630051 1 defaults.go:79] "Error updating the BuildPlatform" err="SonataFlowPlatform.sonataflow.org \"sonataflow-platform\" is invalid: spec.services.jobService.persistence.postgresql: Too many: 3: must have at most 2 items"
```
It's fixed by increasing the `maxProperty` value to 3. @tchughesiv do you know why it was limited to 2 max in the first place? Any reason for the validation? Can it be increased to 3?
* The job service app properties are not added because the code that adds these properties is never called. The code is executed only when the a sonataflow CR is created, but not when the service is created. In this case should it add the same properties to the configmap as it does for a sonataflow CR? Or should it be a subset of those (reactiveURL and something else?). @wmedvede please advise.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "tchughesiv (via GitHub)" <gi...@apache.org>.
tchughesiv commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1427091996
##########
controllers/profiles/common/properties/application_test.go:
##########
@@ -238,3 +182,140 @@ func doTestGenerateMicroprofileServiceCatalogProperty(t *testing.T, serviceUri s
mpProperty := generateMicroprofileServiceCatalogProperty(serviceUri)
assert.Equal(t, mpProperty, expectedProperty, "expected microprofile service catalog property for serviceUri: %s, is %s, but the returned value was: %s", serviceUri, expectedProperty, mpProperty)
}
+
+func Test_appPropertyHandler_WithServicesWithUserOverrides(t *testing.T) {
+ //try to override kogito.service.url and quarkus.http.port
+ userProperties := "property1=value1\nproperty2=value2\nquarkus.http.port=9090\nkogito.service.url=http://myUrl.override.com\nquarkus.http.port=9090"
+ ns := "default"
+ workflow := test.GetBaseSonataFlow(ns)
+ workflow.SetAnnotations(map[string]string{metadata.Profile: string(metadata.DevProfile)})
+ enabled := true
+ platform := test.GetBasePlatform()
+ platform.Namespace = ns
+ platform.Spec = operatorapi.SonataFlowPlatformSpec{
+ Services: operatorapi.ServicesPlatformSpec{
+ DataIndex: &operatorapi.ServiceSpec{
+ Enabled: &enabled,
+ },
+ JobService: &operatorapi.ServiceSpec{
+ Enabled: &enabled,
+ },
+ },
+ }
+
+ di := services.NewDataIndexService(platform)
+
+ props, err := NewAppPropertyHandler(workflow, platform)
+ assert.NoError(t, err)
+ generatedProps, propsErr := properties.LoadString(props.WithUserProperties(userProperties).Build())
+ assert.NoError(t, propsErr)
+ assert.Equal(t, 8, len(generatedProps.Keys()))
+ assert.Equal(t, "value1", generatedProps.GetString("property1", ""))
+ assert.Equal(t, "value2", generatedProps.GetString("property2", ""))
+ //kogito.service.url takes the user provided value since it's a default mutable property.
+ assert.Equal(t, "http://myUrl.override.com", generatedProps.GetString("kogito.service.url", ""))
+ //quarkus.http.port remains with the default value since it's immutable.
+ assert.Equal(t, "8080", generatedProps.GetString("quarkus.http.port", ""))
+ assert.Equal(t, "0.0.0.0", generatedProps.GetString("quarkus.http.host", ""))
+ assert.Equal(t, "false", generatedProps.GetString("org.kie.kogito.addons.knative.eventing.health-enabled", ""))
+ assert.Equal(t, "false", generatedProps.GetString("quarkus.devservices.enabled", ""))
+ assert.Equal(t, "false", generatedProps.GetString("quarkus.kogito.devservices.enabled", ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.DataIndexServiceURLProperty, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceURLProperty, ""))
+
+ // prod profile enables config of outgoing events url
+ workflow.SetAnnotations(map[string]string{metadata.Profile: string(metadata.ProdProfile)})
+ props, err = NewAppPropertyHandler(workflow, platform)
+ assert.NoError(t, err)
+ generatedProps, propsErr = properties.LoadString(props.WithUserProperties(userProperties).Build())
+ assert.NoError(t, propsErr)
+ assert.Equal(t, 13, len(generatedProps.Keys()))
+ assert.Equal(t, "http://"+platform.Name+"-"+constants.DataIndexServiceName+"."+platform.Namespace+"/processes", generatedProps.GetString(constants.DataIndexServiceURLProperty, ""))
+ assert.Equal(t, "http://"+platform.Name+"-"+constants.JobServiceName+"."+platform.Namespace+"/v2/jobs/events", generatedProps.GetString(constants.JobServiceURLProperty, ""))
+ assert.Equal(t, "false", generatedProps.GetString(constants.JobServiceKafkaSinkInjectionHealthCheck, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceDataSourceReactiveURLProperty, ""))
+ assert.Equal(t, "true", generatedProps.GetString(constants.JobServiceStatusChangeEventsProperty, ""))
+ assert.Equal(t, generatedProps.GetString(constants.JobServiceStatusChangeEventsURL, ""), fmt.Sprintf("%s://%s.%s/jobs", constants.DataIndexServiceURLProtocol, di.GetServiceName(), platform.Namespace))
+
+ // disabling data index bypasses config of outgoing events url
+ platform.Spec.Services.DataIndex.Enabled = nil
+ props, err = NewAppPropertyHandler(workflow, platform)
+ assert.NoError(t, err)
+ generatedProps, propsErr = properties.LoadString(props.WithUserProperties(userProperties).Build())
+ assert.NoError(t, propsErr)
+ assert.Equal(t, 10, len(generatedProps.Keys()))
+ assert.Equal(t, "", generatedProps.GetString(constants.DataIndexServiceURLProperty, ""))
+ assert.Equal(t, "http://"+platform.Name+"-"+constants.JobServiceName+"."+platform.Namespace+"/v2/jobs/events", generatedProps.GetString(constants.JobServiceURLProperty, ""))
+ assert.Equal(t, "false", generatedProps.GetString(constants.JobServiceKafkaSinkInjectionHealthCheck, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceStatusChangeEventsProperty, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceStatusChangeEventsURL, ""))
+
+ // disabling job service bypasses config of outgoing events url
+ platform.Spec.Services.JobService.Enabled = nil
+ props, err = NewAppPropertyHandler(workflow, platform)
+ assert.NoError(t, err)
+ generatedProps, propsErr = properties.LoadString(props.WithUserProperties(userProperties).Build())
+ assert.NoError(t, propsErr)
+ assert.Equal(t, 8, len(generatedProps.Keys()))
+ assert.Equal(t, "", generatedProps.GetString(constants.DataIndexServiceURLProperty, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceURLProperty, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceKafkaSinkInjectionHealthCheck, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceDataSourceReactiveURLProperty, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceStatusChangeEventsProperty, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceStatusChangeEventsURL, ""))
+
+ // check that service app properties are being properly set
+ generatedProps, propsErr = properties.LoadString(NewServiceAppPropertyHandler(platform).WithUserProperties(userProperties).Build())
Review Comment:
> there are no DI or JS properties that requires user intervention
while there may not be properties that "require" user intervention... in my opinion, the operator shouldn't be in the business of limiting the user wrt property configs either. we already allow the user the modify just about everything in the DI & JS PodSpec... application properties should be no different. this kind of functionality allows for greater adoption and success in future production-like envs.
##########
controllers/profiles/common/properties/application_test.go:
##########
@@ -238,3 +182,140 @@ func doTestGenerateMicroprofileServiceCatalogProperty(t *testing.T, serviceUri s
mpProperty := generateMicroprofileServiceCatalogProperty(serviceUri)
assert.Equal(t, mpProperty, expectedProperty, "expected microprofile service catalog property for serviceUri: %s, is %s, but the returned value was: %s", serviceUri, expectedProperty, mpProperty)
}
+
+func Test_appPropertyHandler_WithServicesWithUserOverrides(t *testing.T) {
+ //try to override kogito.service.url and quarkus.http.port
+ userProperties := "property1=value1\nproperty2=value2\nquarkus.http.port=9090\nkogito.service.url=http://myUrl.override.com\nquarkus.http.port=9090"
+ ns := "default"
+ workflow := test.GetBaseSonataFlow(ns)
+ workflow.SetAnnotations(map[string]string{metadata.Profile: string(metadata.DevProfile)})
+ enabled := true
+ platform := test.GetBasePlatform()
+ platform.Namespace = ns
+ platform.Spec = operatorapi.SonataFlowPlatformSpec{
+ Services: operatorapi.ServicesPlatformSpec{
+ DataIndex: &operatorapi.ServiceSpec{
+ Enabled: &enabled,
+ },
+ JobService: &operatorapi.ServiceSpec{
+ Enabled: &enabled,
+ },
+ },
+ }
+
+ di := services.NewDataIndexService(platform)
+
+ props, err := NewAppPropertyHandler(workflow, platform)
+ assert.NoError(t, err)
+ generatedProps, propsErr := properties.LoadString(props.WithUserProperties(userProperties).Build())
+ assert.NoError(t, propsErr)
+ assert.Equal(t, 8, len(generatedProps.Keys()))
+ assert.Equal(t, "value1", generatedProps.GetString("property1", ""))
+ assert.Equal(t, "value2", generatedProps.GetString("property2", ""))
+ //kogito.service.url takes the user provided value since it's a default mutable property.
+ assert.Equal(t, "http://myUrl.override.com", generatedProps.GetString("kogito.service.url", ""))
+ //quarkus.http.port remains with the default value since it's immutable.
+ assert.Equal(t, "8080", generatedProps.GetString("quarkus.http.port", ""))
+ assert.Equal(t, "0.0.0.0", generatedProps.GetString("quarkus.http.host", ""))
+ assert.Equal(t, "false", generatedProps.GetString("org.kie.kogito.addons.knative.eventing.health-enabled", ""))
+ assert.Equal(t, "false", generatedProps.GetString("quarkus.devservices.enabled", ""))
+ assert.Equal(t, "false", generatedProps.GetString("quarkus.kogito.devservices.enabled", ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.DataIndexServiceURLProperty, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceURLProperty, ""))
+
+ // prod profile enables config of outgoing events url
+ workflow.SetAnnotations(map[string]string{metadata.Profile: string(metadata.ProdProfile)})
+ props, err = NewAppPropertyHandler(workflow, platform)
+ assert.NoError(t, err)
+ generatedProps, propsErr = properties.LoadString(props.WithUserProperties(userProperties).Build())
+ assert.NoError(t, propsErr)
+ assert.Equal(t, 13, len(generatedProps.Keys()))
+ assert.Equal(t, "http://"+platform.Name+"-"+constants.DataIndexServiceName+"."+platform.Namespace+"/processes", generatedProps.GetString(constants.DataIndexServiceURLProperty, ""))
+ assert.Equal(t, "http://"+platform.Name+"-"+constants.JobServiceName+"."+platform.Namespace+"/v2/jobs/events", generatedProps.GetString(constants.JobServiceURLProperty, ""))
+ assert.Equal(t, "false", generatedProps.GetString(constants.JobServiceKafkaSinkInjectionHealthCheck, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceDataSourceReactiveURLProperty, ""))
+ assert.Equal(t, "true", generatedProps.GetString(constants.JobServiceStatusChangeEventsProperty, ""))
+ assert.Equal(t, generatedProps.GetString(constants.JobServiceStatusChangeEventsURL, ""), fmt.Sprintf("%s://%s.%s/jobs", constants.DataIndexServiceURLProtocol, di.GetServiceName(), platform.Namespace))
+
+ // disabling data index bypasses config of outgoing events url
+ platform.Spec.Services.DataIndex.Enabled = nil
+ props, err = NewAppPropertyHandler(workflow, platform)
+ assert.NoError(t, err)
+ generatedProps, propsErr = properties.LoadString(props.WithUserProperties(userProperties).Build())
+ assert.NoError(t, propsErr)
+ assert.Equal(t, 10, len(generatedProps.Keys()))
+ assert.Equal(t, "", generatedProps.GetString(constants.DataIndexServiceURLProperty, ""))
+ assert.Equal(t, "http://"+platform.Name+"-"+constants.JobServiceName+"."+platform.Namespace+"/v2/jobs/events", generatedProps.GetString(constants.JobServiceURLProperty, ""))
+ assert.Equal(t, "false", generatedProps.GetString(constants.JobServiceKafkaSinkInjectionHealthCheck, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceStatusChangeEventsProperty, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceStatusChangeEventsURL, ""))
+
+ // disabling job service bypasses config of outgoing events url
+ platform.Spec.Services.JobService.Enabled = nil
+ props, err = NewAppPropertyHandler(workflow, platform)
+ assert.NoError(t, err)
+ generatedProps, propsErr = properties.LoadString(props.WithUserProperties(userProperties).Build())
+ assert.NoError(t, propsErr)
+ assert.Equal(t, 8, len(generatedProps.Keys()))
+ assert.Equal(t, "", generatedProps.GetString(constants.DataIndexServiceURLProperty, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceURLProperty, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceKafkaSinkInjectionHealthCheck, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceDataSourceReactiveURLProperty, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceStatusChangeEventsProperty, ""))
+ assert.Equal(t, "", generatedProps.GetString(constants.JobServiceStatusChangeEventsURL, ""))
+
+ // check that service app properties are being properly set
+ generatedProps, propsErr = properties.LoadString(NewServiceAppPropertyHandler(platform).WithUserProperties(userProperties).Build())
Review Comment:
> there are no DI or JS properties that requires user intervention
while there may not be properties that "require" user intervention... in my opinion, the operator shouldn't be in the business of limiting the user wrt property configs either. we already allow the user the modify just about everything in the DI & JS PodSpec... application properties should be no different. this kind of functionality allows for greater adoption and success in future production-like envs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1418289714
##########
controllers/platform/services.go:
##########
@@ -99,27 +98,9 @@ func createDataIndexDeployment(ctx context.Context, client client.Client, platfo
liveProbe := readyProbe.DeepCopy()
liveProbe.ProbeHandler.HTTPGet.Path = common.QuarkusHealthPathLive
dataDeployContainer := &corev1.Container{
- Image: common.DataIndexImageBase + common.PersistenceTypeEphemeral,
- Env: []corev1.EnvVar{
- {
- Name: "KOGITO_DATA_INDEX_QUARKUS_PROFILE",
- Value: "http-events-support",
- },
- {
- Name: "QUARKUS_HTTP_CORS",
- Value: "true",
- },
- {
- Name: "QUARKUS_HTTP_CORS_ORIGINS",
- Value: "/.*/",
- },
- },
- Resources: corev1.ResourceRequirements{
- Limits: corev1.ResourceList{
- corev1.ResourceCPU: resource.MustParse("100m"),
- corev1.ResourceMemory: resource.MustParse("256Mi"),
- },
- },
+ Image: svc.GetServiceImageName(common.PersistenceTypeEphemeral),
+ Env: svc.GetEnvironmentVariables(),
+ Resources: svc.GetPodResourceRequirements(),
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1419462954
##########
controllers/platform/services/properties.go:
##########
@@ -0,0 +1,97 @@
+// Copyright 2023 Apache Software Foundation (ASF)
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package services
+
+import (
+ "fmt"
+ "strings"
+
+ operatorapi "github.com/apache/incubator-kie-kogito-serverless-operator/api/v1alpha08"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles/common/constants"
+
+ "github.com/magiconair/properties"
+)
+
+func generateReactiveURL(postgresSpec *operatorapi.PersistencePostgreSql, schema string, namespace string, dbName string, port int) string {
+ if len(postgresSpec.JdbcUrl) > 0 && strings.Contains(postgresSpec.JdbcUrl, "currentSchema=") {
+ return strings.Replace(strings.TrimPrefix(postgresSpec.JdbcUrl, "jdbc:"), "currentSchema=", "search_path=", -1)
+ }
+ databaseSchema := schema
+ if len(postgresSpec.ServiceRef.DatabaseSchema) > 0 {
+ databaseSchema = postgresSpec.ServiceRef.DatabaseSchema
+ }
+ databaseNamespace := namespace
+ if len(postgresSpec.ServiceRef.Namespace) > 0 {
+ databaseNamespace = postgresSpec.ServiceRef.Namespace
+ }
+ dataSourcePort := port
+ if postgresSpec.ServiceRef.Port != nil {
+ dataSourcePort = *postgresSpec.ServiceRef.Port
+ }
+ databaseName := dbName
+ if len(postgresSpec.ServiceRef.DatabaseName) > 0 {
+ databaseName = postgresSpec.ServiceRef.DatabaseName
+ }
+ return fmt.Sprintf("%s://%s:%d/%s?search_path=%s", constants.PersistenceTypePostgreSQL, postgresSpec.ServiceRef.Name+"."+databaseNamespace, dataSourcePort, databaseName, databaseSchema)
+}
+
+// withDataIndexServiceUrl adds the property dataIndexServiceUrlProperty to the application properties.
+// See Service Discovery https://kubernetes.io/docs/concepts/services-networking/service/#dns
+func GenerateDataIndexApplicationProperties(workflow *operatorapi.SonataFlow, platform *operatorapi.SonataFlowPlatform) *properties.Properties {
Review Comment:
Done
##########
controllers/platform/services/properties.go:
##########
@@ -0,0 +1,97 @@
+// Copyright 2023 Apache Software Foundation (ASF)
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package services
+
+import (
+ "fmt"
+ "strings"
+
+ operatorapi "github.com/apache/incubator-kie-kogito-serverless-operator/api/v1alpha08"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles/common/constants"
+
+ "github.com/magiconair/properties"
+)
+
+func generateReactiveURL(postgresSpec *operatorapi.PersistencePostgreSql, schema string, namespace string, dbName string, port int) string {
+ if len(postgresSpec.JdbcUrl) > 0 && strings.Contains(postgresSpec.JdbcUrl, "currentSchema=") {
+ return strings.Replace(strings.TrimPrefix(postgresSpec.JdbcUrl, "jdbc:"), "currentSchema=", "search_path=", -1)
+ }
+ databaseSchema := schema
+ if len(postgresSpec.ServiceRef.DatabaseSchema) > 0 {
+ databaseSchema = postgresSpec.ServiceRef.DatabaseSchema
+ }
+ databaseNamespace := namespace
+ if len(postgresSpec.ServiceRef.Namespace) > 0 {
+ databaseNamespace = postgresSpec.ServiceRef.Namespace
+ }
+ dataSourcePort := port
+ if postgresSpec.ServiceRef.Port != nil {
+ dataSourcePort = *postgresSpec.ServiceRef.Port
+ }
+ databaseName := dbName
+ if len(postgresSpec.ServiceRef.DatabaseName) > 0 {
+ databaseName = postgresSpec.ServiceRef.DatabaseName
+ }
+ return fmt.Sprintf("%s://%s:%d/%s?search_path=%s", constants.PersistenceTypePostgreSQL, postgresSpec.ServiceRef.Name+"."+databaseNamespace, dataSourcePort, databaseName, databaseSchema)
+}
+
+// withDataIndexServiceUrl adds the property dataIndexServiceUrlProperty to the application properties.
+// See Service Discovery https://kubernetes.io/docs/concepts/services-networking/service/#dns
+func GenerateDataIndexApplicationProperties(workflow *operatorapi.SonataFlow, platform *operatorapi.SonataFlowPlatform) *properties.Properties {
+ props := properties.NewProperties()
+ if profiles.IsProdProfile(workflow) && dataIndexEnabled(platform) {
+ di := NewDataIndexService(platform)
+ props.Set(
+ constants.DataIndexServiceURLProperty,
+ fmt.Sprintf("%s://%s.%s/processes", constants.DataIndexServiceURLProtocol, di.GetServiceName(), platform.Namespace),
+ )
+ }
+ return props
+}
+
+// withJobServiceURL adds the property 'mp.messaging.outgoing.kogito-job-service-job-request-events.url' to the application property
+func GenerateJobServiceApplicationProperties(workflow *operatorapi.SonataFlow, platform *operatorapi.SonataFlowPlatform) *properties.Properties {
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1424840988
##########
controllers/platform/services/services.go:
##########
@@ -0,0 +1,298 @@
+// Copyright 2023 Red Hat, Inc. and/or its affiliates
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package services
+
+import (
+ "fmt"
+ "strconv"
+
+ corev1 "k8s.io/api/core/v1"
+ "k8s.io/apimachinery/pkg/api/resource"
+
+ operatorapi "github.com/apache/incubator-kie-kogito-serverless-operator/api/v1alpha08"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles/common/constants"
+
+ "github.com/apache/incubator-kie-kogito-serverless-operator/version"
+ "github.com/imdario/mergo"
+)
+
+type Platform interface {
+ // GetContainerName returns the name of the service's container in the deployment.
+ GetContainerName() string
+ // GetServiceImageName returns the image name of the service's container. It takes in the service and persistence types and returns a string
+ // that contains the FQDN of the image, including the tag.
+ GetServiceImageName(persistenceName string) string
+ // GetServiceName returns the name of the kubernetes service prefixed with the platform name
+ GetServiceName() string
+ // GetServiceCmName returns the name of the configmap associated to the service
+ GetServiceCmName() string
+ // GetEnvironmentVariables returns the env variables to be injected to the service container
+ GetEnvironmentVariables() []corev1.EnvVar
+ // GetResourceLimits returns the pod's memory and CPU resource requirements
+ // Values for job service taken from
+ // https://github.com/parodos-dev/orchestrator-helm-chart/blob/52d09eda56fdbed3060782df29847c97f172600f/charts/orchestrator/values.yaml#L68-L72
+ GetPodResourceRequirements() corev1.ResourceRequirements
+ // GetReplicaCountForService Returns the default pod replica count for the given service
+ GetReplicaCount() int32
Review Comment:
Done, hardcoded to return always 1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1424401997
##########
controllers/platform/services/services.go:
##########
@@ -0,0 +1,298 @@
+// Copyright 2023 Red Hat, Inc. and/or its affiliates
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package services
+
+import (
+ "fmt"
+ "strconv"
+
+ corev1 "k8s.io/api/core/v1"
+ "k8s.io/apimachinery/pkg/api/resource"
+
+ operatorapi "github.com/apache/incubator-kie-kogito-serverless-operator/api/v1alpha08"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles/common/constants"
+
+ "github.com/apache/incubator-kie-kogito-serverless-operator/version"
+ "github.com/imdario/mergo"
+)
+
+type Platform interface {
+ // GetContainerName returns the name of the service's container in the deployment.
+ GetContainerName() string
+ // GetServiceImageName returns the image name of the service's container. It takes in the service and persistence types and returns a string
+ // that contains the FQDN of the image, including the tag.
+ GetServiceImageName(persistenceName string) string
+ // GetServiceName returns the name of the kubernetes service prefixed with the platform name
+ GetServiceName() string
+ // GetServiceCmName returns the name of the configmap associated to the service
+ GetServiceCmName() string
+ // GetEnvironmentVariables returns the env variables to be injected to the service container
+ GetEnvironmentVariables() []corev1.EnvVar
+ // GetResourceLimits returns the pod's memory and CPU resource requirements
+ // Values for job service taken from
+ // https://github.com/parodos-dev/orchestrator-helm-chart/blob/52d09eda56fdbed3060782df29847c97f172600f/charts/orchestrator/values.yaml#L68-L72
+ GetPodResourceRequirements() corev1.ResourceRequirements
+ // GetReplicaCountForService Returns the default pod replica count for the given service
+ GetReplicaCount() int32
+
+ // MergeContainerSpec performs a merge with override using the containerSpec argument and the expected values based on the service's pod template specifications. The returning
+ // object is the merged result
+ MergeContainerSpec(containerSpec *corev1.Container) (*corev1.Container, error)
+
+ //ConfigurePersistence sets the persistence's image and environment values when it is defined in the Persistence field of the service, overriding any existing value.
+ ConfigurePersistence(containerSpec *corev1.Container) *corev1.Container
+
+ //MergePodSpec performs a merge with override between the podSpec argument and the expected values based on the service's pod template specification. The returning
+ // object is the result of the merge
+ MergePodSpec(podSpec corev1.PodSpec) (corev1.PodSpec, error)
+}
+
+type DataIndex struct {
+ platform *operatorapi.SonataFlowPlatform
+}
+
+func NewDataIndexService(platform *operatorapi.SonataFlowPlatform) DataIndex {
+ return DataIndex{platform: platform}
+}
+
+func (d DataIndex) GetContainerName() string {
+ return constants.DataIndexServiceName
+}
+
+func (d DataIndex) GetServiceImageName(persistenceName string) string {
+ var tag = version.GetMajorMinor()
+ if version.IsSnapshot() {
+ tag = "latest"
+ }
+ // returns "quay.io/kiegroup/kogito-data-index-<persistence_layer>:<tag>"
+ return fmt.Sprintf("%s-%s-%s:%s", constants.ImageNamePrefix, constants.DataIndexName, persistenceName, tag)
+}
+
+func (d DataIndex) GetServiceName() string {
+ return fmt.Sprintf("%s-%s", d.platform.Name, constants.DataIndexServiceName)
+}
+
+func (d DataIndex) GetEnvironmentVariables() []corev1.EnvVar {
+ return []corev1.EnvVar{
+ {
+ Name: "KOGITO_DATA_INDEX_QUARKUS_PROFILE",
+ Value: "http-events-support",
+ },
+ {
+ Name: "QUARKUS_HTTP_CORS",
+ Value: "true",
+ },
+ {
+ Name: "QUARKUS_HTTP_CORS_ORIGINS",
+ Value: "/.*/",
+ },
+ }
+}
+
+func (d DataIndex) GetPodResourceRequirements() corev1.ResourceRequirements {
+ return corev1.ResourceRequirements{
+ Limits: corev1.ResourceList{
+ corev1.ResourceCPU: resource.MustParse("100m"),
+ corev1.ResourceMemory: resource.MustParse("256Mi"),
+ },
+ }
+}
+
+func (d DataIndex) MergePodSpec(podSpec corev1.PodSpec) (corev1.PodSpec, error) {
+ c := podSpec.DeepCopy()
+ err := mergo.Merge(c, d.platform.Spec.Services.DataIndex.PodTemplate.PodSpec.ToPodSpec(), mergo.WithOverride)
+ return *c, err
+}
+
+func (d DataIndex) ConfigurePersistence(containerSpec *corev1.Container) *corev1.Container {
+
+ if d.platform.Spec.Services.DataIndex.Persistence != nil && d.platform.Spec.Services.DataIndex.Persistence.PostgreSql != nil {
+ c := containerSpec.DeepCopy()
+ c.Image = d.GetServiceImageName(constants.PersistenceTypePostgreSQL)
+ c.Env = append(c.Env, configurePostgreSqlEnv(d.platform.Spec.Services.DataIndex.Persistence.PostgreSql, d.GetServiceName(), d.platform.Namespace)...)
+ return c
+ }
+ return containerSpec
+}
+
+func (d DataIndex) MergeContainerSpec(containerSpec *corev1.Container) (*corev1.Container, error) {
+ c := containerSpec.DeepCopy()
+ err := mergo.Merge(c, d.platform.Spec.Services.DataIndex.PodTemplate.Container.ToContainer(), mergo.WithOverride)
+ return c, err
+}
+
+func (d DataIndex) GetReplicaCount() int32 {
+ if d.platform.Spec.Services.DataIndex.PodTemplate.Replicas != nil {
+ return *d.platform.Spec.Services.DataIndex.PodTemplate.Replicas
+ }
+ return 1
+}
+
+func (d DataIndex) GetServiceCmName() string {
+ return fmt.Sprintf("%s-props", d.GetServiceName())
+}
+
+type JobService struct {
+ platform *operatorapi.SonataFlowPlatform
+}
+
+func NewJobService(platform *operatorapi.SonataFlowPlatform) JobService {
+ return JobService{platform: platform}
+}
+
+func (j JobService) GetContainerName() string {
+ return constants.JobServiceName
+}
+
+func (j JobService) GetServiceImageName(persistenceName string) string {
+ var tag = version.GetMajorMinor()
+ if version.IsSnapshot() {
+ tag = "latest"
+ }
+ // returns "quay.io/kiegroup/kogito-jobs-service-<persistece_layer>:<tag>"
+ return fmt.Sprintf("%s-%s-%s:%s", constants.ImageNamePrefix, constants.JobServiceName, persistenceName, tag)
+}
+
+func (j JobService) GetServiceName() string {
+ return fmt.Sprintf("%s-%s", j.platform.Name, constants.JobServiceName)
+}
+
+func (j JobService) GetServiceCmName() string {
+ return fmt.Sprintf("%s-props", j.GetServiceName())
+}
+
+func (j JobService) GetEnvironmentVariables() []corev1.EnvVar {
+ return []corev1.EnvVar{
+ {
+ Name: "QUARKUS_HTTP_CORS",
+ Value: "true",
+ },
+ {
+ Name: "QUARKUS_HTTP_CORS_ORIGINS",
+ Value: "/.*/",
+ },
+ }
+}
+
+func (j JobService) GetPodResourceRequirements() corev1.ResourceRequirements {
+ return corev1.ResourceRequirements{
+ Requests: corev1.ResourceList{
+ corev1.ResourceCPU: resource.MustParse("250m"),
+ corev1.ResourceMemory: resource.MustParse("64Mi"),
+ },
+ Limits: corev1.ResourceList{
+ corev1.ResourceCPU: resource.MustParse("500m"),
+ corev1.ResourceMemory: resource.MustParse("1Gi"),
+ },
+ }
+}
+
+func (j JobService) GetReplicaCount() int32 {
+ if j.platform.Spec.Services.JobService.PodTemplate.Replicas != nil {
+ return *j.platform.Spec.Services.JobService.PodTemplate.Replicas
+ }
+ return 1
+}
+
+func (j JobService) MergeContainerSpec(containerSpec *corev1.Container) (*corev1.Container, error) {
+ c := containerSpec.DeepCopy()
+ err := mergo.Merge(c, j.platform.Spec.Services.JobService.PodTemplate.Container.ToContainer(), mergo.WithOverride)
+ return c, err
+}
+
+func (j JobService) ConfigurePersistence(containerSpec *corev1.Container) *corev1.Container {
+
+ if j.platform.Spec.Services.JobService.Persistence != nil && j.platform.Spec.Services.JobService.Persistence.PostgreSql != nil {
+ c := containerSpec.DeepCopy()
+ c.Image = j.GetServiceImageName(constants.PersistenceTypePostgreSQL)
+ c.Env = append(c.Env, configurePostgreSqlEnv(j.platform.Spec.Services.JobService.Persistence.PostgreSql, j.GetServiceName(), j.platform.Namespace)...)
+ return c
+ }
+ return containerSpec
+}
+
+func (j JobService) MergePodSpec(podSpec corev1.PodSpec) (corev1.PodSpec, error) {
+ c := podSpec.DeepCopy()
+ err := mergo.Merge(c, j.platform.Spec.Services.JobService.PodTemplate.PodSpec.ToPodSpec(), mergo.WithOverride)
+ return *c, err
+}
+
+func configurePostgreSqlEnv(postgresql *operatorapi.PersistencePostgreSql, databaseSchema, databaseNamespace string) []corev1.EnvVar {
+ if len(postgresql.ServiceRef.DatabaseSchema) > 0 {
+ databaseSchema = postgresql.ServiceRef.DatabaseSchema
+ }
+ if len(postgresql.ServiceRef.Namespace) > 0 {
+ databaseNamespace = postgresql.ServiceRef.Namespace
+ }
+ dataSourcePort := 5432
+ if postgresql.ServiceRef.Port != nil {
+ dataSourcePort = *postgresql.ServiceRef.Port
+ }
+ databaseName := "sonataflow"
+ if len(postgresql.ServiceRef.DatabaseName) > 0 {
+ databaseName = postgresql.ServiceRef.DatabaseName
+ }
+ dataSourceUrl := "jdbc:" + constants.PersistenceTypePostgreSQL + "://" + postgresql.ServiceRef.Name + "." + databaseNamespace + ":" + strconv.Itoa(dataSourcePort) + "/" + databaseName + "?currentSchema=" + databaseSchema
+ if len(postgresql.JdbcUrl) > 0 {
+ dataSourceUrl = postgresql.JdbcUrl
+ }
+ secretRef := corev1.LocalObjectReference{
+ Name: postgresql.SecretRef.Name,
+ }
+ quarkusDatasourceUsername := "POSTGRESQL_USER"
+ if len(postgresql.SecretRef.UserKey) > 0 {
+ quarkusDatasourceUsername = postgresql.SecretRef.UserKey
+ }
+ quarkusDatasourcePassword := "POSTGRESQL_PASSWORD"
+ if len(postgresql.SecretRef.PasswordKey) > 0 {
+ quarkusDatasourcePassword = postgresql.SecretRef.PasswordKey
+ }
+ return []corev1.EnvVar{
+ {
+ Name: "QUARKUS_DATASOURCE_USERNAME",
+ ValueFrom: &corev1.EnvVarSource{
+ SecretKeyRef: &corev1.SecretKeySelector{
+ Key: quarkusDatasourceUsername,
+ LocalObjectReference: secretRef,
+ },
+ },
+ },
+ {
+ Name: "QUARKUS_DATASOURCE_PASSWORD",
+ ValueFrom: &corev1.EnvVarSource{
+ SecretKeyRef: &corev1.SecretKeySelector{
+ Key: quarkusDatasourcePassword,
+ LocalObjectReference: secretRef,
+ },
+ },
+ },
+ {
+ Name: "QUARKUS_DATASOURCE_DB_KIND",
+ Value: constants.PersistenceTypePostgreSQL,
+ },
+ {
+ Name: "QUARKUS_HIBERNATE_ORM_DATABASE_GENERATION",
Review Comment:
ack
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1424402245
##########
controllers/profiles/common/constants/platform_services.go:
##########
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package constants
+
+const (
+ ConfigMapWorkflowPropsVolumeName = "workflow-properties"
+ PersistenceTypePostgreSQL = "postgresql"
+ PersistenceTypeEphemeral = "ephemeral"
+
+ DataIndexServiceURLProperty = "mp.messaging.outgoing.kogito-processinstances-events.url"
+ DataIndexServiceURLProtocol = "http"
+
+ JobServiceURLProperty = "mp.messaging.outgoing.kogito-job-service-job-request-events.url"
+ JobServiceKafkaSinkInjectionHealthCheck = `quarkus.smallrye-health.check."org.kie.kogito.jobs.service.messaging.http.health.knative.KSinkInjectionHealthCheck".enabled`
+ JobServiceStatusChangeEventsProperty = "kogito.jobs-service.http.job-status-change-events"
+ JobServiceStatusChangeEventsURL = "mp.messaging.outgoing.kogito-job-service-job-status-events-http.url"
+ JobServiceURLProtocol = "http"
+ JobServiceDataSourceReactiveURLProperty = "quarkus.datasource.reactive.url"
+
+ KogitoServiceUrlProperty = "kogito.service.url"
+ KogitoServiceUrlProtocol = "http"
+ KafkaSmallRyeHealthProperty = "quarkus.smallrye-health.check.\"io.quarkus.kafka.client.health.KafkaHealthCheck\".enabled"
+
+ MicroprofileServiceCatalogPropertyPrefix = "org.kie.kogito.addons.discovery."
Review Comment:
ack
##########
controllers/profiles/common/constants/platform_services.go:
##########
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package constants
+
+const (
+ ConfigMapWorkflowPropsVolumeName = "workflow-properties"
+ PersistenceTypePostgreSQL = "postgresql"
+ PersistenceTypeEphemeral = "ephemeral"
+
+ DataIndexServiceURLProperty = "mp.messaging.outgoing.kogito-processinstances-events.url"
+ DataIndexServiceURLProtocol = "http"
+
+ JobServiceURLProperty = "mp.messaging.outgoing.kogito-job-service-job-request-events.url"
+ JobServiceKafkaSinkInjectionHealthCheck = `quarkus.smallrye-health.check."org.kie.kogito.jobs.service.messaging.http.health.knative.KSinkInjectionHealthCheck".enabled`
+ JobServiceStatusChangeEventsProperty = "kogito.jobs-service.http.job-status-change-events"
+ JobServiceStatusChangeEventsURL = "mp.messaging.outgoing.kogito-job-service-job-status-events-http.url"
+ JobServiceURLProtocol = "http"
+ JobServiceDataSourceReactiveURLProperty = "quarkus.datasource.reactive.url"
+
+ KogitoServiceUrlProperty = "kogito.service.url"
+ KogitoServiceUrlProtocol = "http"
+ KafkaSmallRyeHealthProperty = "quarkus.smallrye-health.check.\"io.quarkus.kafka.client.health.KafkaHealthCheck\".enabled"
+
+ MicroprofileServiceCatalogPropertyPrefix = "org.kie.kogito.addons.discovery."
+ DiscoveryLikePropertyPattern = "^\\${(kubernetes|knative|openshift):(.*)}$"
Review Comment:
ack
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "wmedvede (via GitHub)" <gi...@apache.org>.
wmedvede commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1424979529
##########
controllers/platform/services/properties.go:
##########
@@ -0,0 +1,98 @@
+// Copyright 2023 Apache Software Foundation (ASF)
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package services
+
+import (
+ "fmt"
+ "strings"
+
+ operatorapi "github.com/apache/incubator-kie-kogito-serverless-operator/api/v1alpha08"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles/common/constants"
+
+ "github.com/magiconair/properties"
+)
+
+func generateReactiveURL(postgresSpec *operatorapi.PersistencePostgreSql, schema string, namespace string, dbName string, port int) string {
+ if len(postgresSpec.JdbcUrl) > 0 && strings.Contains(postgresSpec.JdbcUrl, "currentSchema=") {
+ return strings.Replace(strings.TrimPrefix(postgresSpec.JdbcUrl, "jdbc:"), "currentSchema=", "search_path=", -1)
Review Comment:
yes, current schema is optional, and if no currentSchema, no search_path
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "tchughesiv (via GitHub)" <gi...@apache.org>.
tchughesiv commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1425316048
##########
controllers/profiles/common/properties/application.go:
##########
@@ -143,31 +131,18 @@ func (a *appPropertyHandler) Build() string {
func (a *appPropertyHandler) withKogitoServiceUrl() AppPropertyHandler {
var kogitoServiceUrl string
if len(a.workflow.Namespace) > 0 {
- kogitoServiceUrl = fmt.Sprintf("%s://%s.%s", kogitoServiceUrlProtocol, a.workflow.Name, a.workflow.Namespace)
+ kogitoServiceUrl = fmt.Sprintf("%s://%s.%s", constants.KogitoServiceUrlProtocol, a.workflow.Name, a.workflow.Namespace)
} else {
- kogitoServiceUrl = fmt.Sprintf("%s://%s", kogitoServiceUrlProtocol, a.workflow.Name)
+ kogitoServiceUrl = fmt.Sprintf("%s://%s", constants.KogitoServiceUrlProtocol, a.workflow.Name)
}
- return a.addDefaultMutableProperty(kogitoServiceUrlProperty, kogitoServiceUrl)
-}
-
-// withDataIndexServiceUrl adds the property dataIndexServiceUrlProperty to the application properties.
-// See Service Discovery https://kubernetes.io/docs/concepts/services-networking/service/#dns
-func (a *appPropertyHandler) withDataIndexServiceUrl() AppPropertyHandler {
- if profiles.IsProdProfile(a.workflow) && dataIndexEnabled(a.platform) {
Review Comment:
why is the for "prod" profile check being removed from this property handler? @ricardozanini wasn't the intention that these services only be used by "prod" workflows? that "dev" workflows should continue to use their embedded services?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1409341906
##########
controllers/profiles/common/app_properties.go:
##########
@@ -156,7 +194,7 @@ func (a *appPropertyHandler) withDataIndexServiceUrl() AppPropertyHandler {
if profiles.IsProdProfile(a.workflow) && dataIndexEnabled(a.platform) {
a.addDefaultMutableProperty(
dataIndexServiceUrlProperty,
- fmt.Sprintf("%s://%s.%s/processes", dataIndexServiceUrlProtocol, GetDataIndexName(a.platform), a.platform.Namespace),
+ fmt.Sprintf("%s://%s.%s/processes", dataIndexServiceUrlProtocol, GetServiceName(a.platform.Name, DataIndexService), a.platform.Namespace),
Review Comment:
Looks like it [has](https://kiegroup.github.io/kogito-docs/serverlessworkflow/latest/job-services/quarkus-extensions.html#kogito-addons-quarkus-jobs-knative-eventing) at least 3 from what I can gather by the data index property.
```
mp.messaging.outgoing.kogito-job-service-job-request-events.connector=quarkus-http
mp.messaging.outgoing.kogito-job-service-job-request-events.url=${K_SINK:http://localhost:8280/v2/jobs/events}
mp.messaging.outgoing.kogito-job-service-job-request-events.method=POST
```
But, as it [happens](https://github.com/apache/incubator-kie-kogito-serverless-operator/blob/162f3693844de04e94b447b29253eb3ed38057ad/controllers/profiles/common/app_properties.go#L44) with data index, only the `URL` property is added in the code for data index, the following being the equivalent for job service:
```
mp.messaging.outgoing.kogito-job-service-job-request-events.url
```
I will follow example from data index and add this last property if the job service is enabled
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "tchughesiv (via GitHub)" <gi...@apache.org>.
tchughesiv commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1409729898
##########
api/v1alpha08/sonataflowplatform_services_types.go:
##########
@@ -19,6 +19,9 @@ type ServicesPlatformSpec struct {
// Deploys the Data Index service for use by "prod" profile workflows.
// +optional
DataIndex *ServiceSpec `json:"dataIndex,omitempty"`
+ // Deploys the Job service for use by "prod" profile workflows.
+ // +optional
+ JobService *ServiceSpec `json:"job,omitempty"`
Review Comment:
didn't we talk about naming this field `jobService` ?
```go
JobService *ServiceSpec `json:"jobService,omitempty"`
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1408285435
##########
controllers/profiles/common/app_properties.go:
##########
@@ -45,27 +45,59 @@ const (
kafkaSmallRyeHealthProperty = "quarkus.smallrye-health.check.\"io.quarkus.kafka.client.health.KafkaHealthCheck\".enabled"
dataIndexServiceUrlProtocol = "http"
- DataIndexImageBase = "quay.io/kiegroup/kogito-data-index-"
- DataIndexName = "data-index-service"
-
- PersistenceTypeEphemeral = "ephemeral"
- PersistenceTypePostgressql = "postgresql"
-
+ imageNamePrefix = "quay.io/kiegroup/kogito"
+ DataIndexServiceName = "data-index-service"
+ DataIndexName = "data-index"
+ JobServiceName = "jobs-service"
+ ImageTag = "1.44"
+ PersistenceTypeEphemeral = "ephemeral"
+ PersistenceTypePostgressql = "postgresql"
microprofileServiceCatalogPropertyPrefix = "org.kie.kogito.addons.discovery."
discoveryLikePropertyPattern = "^\\${(kubernetes|knative|openshift):(.*)}$"
)
-var immutableApplicationProperties = "quarkus.http.port=" + DefaultHTTPWorkflowPortIntStr.String() + "\n" +
- "quarkus.http.host=0.0.0.0\n" +
- // We disable the Knative health checks to not block the dev pod to run if Knative objects are not available
- // See: https://kiegroup.github.io/kogito-docs/serverlessworkflow/latest/eventing/consume-produce-events-with-knative-eventing.html#ref-knative-eventing-add-on-source-configuration
- "org.kie.kogito.addons.knative.eventing.health-enabled=false\n" +
- "quarkus.devservices.enabled=false\n" +
- "quarkus.kogito.devservices.enabled=false\n"
+var (
+ immutableApplicationProperties = "quarkus.http.port=" + DefaultHTTPWorkflowPortIntStr.String() + "\n" +
+ "quarkus.http.host=0.0.0.0\n" +
+ // We disable the Knative health checks to not block the dev pod to run if Knative objects are not available
+ // See: https://kiegroup.github.io/kogito-docs/serverlessworkflow/latest/eventing/consume-produce-events-with-knative-eventing.html#ref-knative-eventing-add-on-source-configuration
+ "org.kie.kogito.addons.knative.eventing.health-enabled=false\n" +
+ "quarkus.devservices.enabled=false\n" +
+ "quarkus.kogito.devservices.enabled=false\n"
+
+ discoveryLikePropertyExpr = regexp.MustCompile(discoveryLikePropertyPattern)
+ _ AppPropertyHandler = &appPropertyHandler{}
+)
-var discoveryLikePropertyExpr = regexp.MustCompile(discoveryLikePropertyPattern)
+const (
+ DataIndexService = iota
+ JobService
+)
-var _ AppPropertyHandler = &appPropertyHandler{}
+// GetContainerName returns the name of the service's container in the deployment.
+func GetContainerName(stype int) string {
Review Comment:
Added enum type for clarity.
##########
controllers/profiles/common/app_properties.go:
##########
@@ -45,27 +45,59 @@ const (
kafkaSmallRyeHealthProperty = "quarkus.smallrye-health.check.\"io.quarkus.kafka.client.health.KafkaHealthCheck\".enabled"
dataIndexServiceUrlProtocol = "http"
- DataIndexImageBase = "quay.io/kiegroup/kogito-data-index-"
- DataIndexName = "data-index-service"
-
- PersistenceTypeEphemeral = "ephemeral"
- PersistenceTypePostgressql = "postgresql"
-
+ imageNamePrefix = "quay.io/kiegroup/kogito"
+ DataIndexServiceName = "data-index-service"
+ DataIndexName = "data-index"
+ JobServiceName = "jobs-service"
+ ImageTag = "1.44"
+ PersistenceTypeEphemeral = "ephemeral"
+ PersistenceTypePostgressql = "postgresql"
microprofileServiceCatalogPropertyPrefix = "org.kie.kogito.addons.discovery."
discoveryLikePropertyPattern = "^\\${(kubernetes|knative|openshift):(.*)}$"
)
-var immutableApplicationProperties = "quarkus.http.port=" + DefaultHTTPWorkflowPortIntStr.String() + "\n" +
- "quarkus.http.host=0.0.0.0\n" +
- // We disable the Knative health checks to not block the dev pod to run if Knative objects are not available
- // See: https://kiegroup.github.io/kogito-docs/serverlessworkflow/latest/eventing/consume-produce-events-with-knative-eventing.html#ref-knative-eventing-add-on-source-configuration
- "org.kie.kogito.addons.knative.eventing.health-enabled=false\n" +
- "quarkus.devservices.enabled=false\n" +
- "quarkus.kogito.devservices.enabled=false\n"
+var (
+ immutableApplicationProperties = "quarkus.http.port=" + DefaultHTTPWorkflowPortIntStr.String() + "\n" +
+ "quarkus.http.host=0.0.0.0\n" +
+ // We disable the Knative health checks to not block the dev pod to run if Knative objects are not available
+ // See: https://kiegroup.github.io/kogito-docs/serverlessworkflow/latest/eventing/consume-produce-events-with-knative-eventing.html#ref-knative-eventing-add-on-source-configuration
+ "org.kie.kogito.addons.knative.eventing.health-enabled=false\n" +
+ "quarkus.devservices.enabled=false\n" +
+ "quarkus.kogito.devservices.enabled=false\n"
+
+ discoveryLikePropertyExpr = regexp.MustCompile(discoveryLikePropertyPattern)
+ _ AppPropertyHandler = &appPropertyHandler{}
+)
-var discoveryLikePropertyExpr = regexp.MustCompile(discoveryLikePropertyPattern)
+const (
+ DataIndexService = iota
+ JobService
+)
-var _ AppPropertyHandler = &appPropertyHandler{}
+// GetContainerName returns the name of the service's container in the deployment.
+func GetContainerName(stype int) string {
+ switch stype {
+ case DataIndexService:
+ return DataIndexServiceName
+ case JobService:
+ return JobServiceName
+ }
+ return ""
+}
+
+// GetServiceImageName returns the image name of the service's container. It takes in the service and persistence types and returns a string
+// that contains the FQDN of the image, including the tag.
+func GetServiceImageName(persistenceName string, stype int) string {
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "tchughesiv (via GitHub)" <gi...@apache.org>.
tchughesiv commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1411092934
##########
controllers/platform/services.go:
##########
@@ -58,30 +60,72 @@ func (action *serviceAction) Handle(ctx context.Context, platform *operatorapi.S
return nil, err
}
- if err := createDataIndexComponents(ctx, action.client, platform); err != nil {
- return nil, err
+ if platform.Spec.Services.DataIndex != nil {
Review Comment:
KOGITO-9972 will require changes to `SonataFlowPlatform.status` and, i expect, will be a good time to make some enhancements.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "masayag (via GitHub)" <gi...@apache.org>.
masayag commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1415998743
##########
controllers/platform/services.go:
##########
@@ -58,30 +55,32 @@ func (action *serviceAction) Handle(ctx context.Context, platform *operatorapi.S
return nil, err
}
- if err := createDataIndexComponents(ctx, action.client, platform); err != nil {
- return nil, err
+ if platform.Spec.Services.DataIndex != nil {
Review Comment:
Should we get into this block if `platform.Spec.Services.DataIndex.Enabled == false`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1418285572
##########
controllers/profiles/common/app_properties.go:
##########
@@ -38,34 +43,313 @@ import (
)
const (
- ConfigMapWorkflowPropsVolumeName = "workflow-properties"
- kogitoServiceUrlProperty = "kogito.service.url"
- kogitoServiceUrlProtocol = "http"
- dataIndexServiceUrlProperty = "mp.messaging.outgoing.kogito-processinstances-events.url"
- kafkaSmallRyeHealthProperty = "quarkus.smallrye-health.check.\"io.quarkus.kafka.client.health.KafkaHealthCheck\".enabled"
- dataIndexServiceUrlProtocol = "http"
-
- DataIndexImageBase = "quay.io/kiegroup/kogito-data-index-"
- DataIndexName = "data-index-service"
-
- PersistenceTypeEphemeral = "ephemeral"
- PersistenceTypePostgressql = "postgresql"
-
+ ConfigMapWorkflowPropsVolumeName = "workflow-properties"
+ kogitoServiceUrlProperty = "kogito.service.url"
+ kogitoServiceUrlProtocol = "http"
+ dataIndexServiceUrlProperty = "mp.messaging.outgoing.kogito-processinstances-events.url"
+ kafkaSmallRyeHealthProperty = "quarkus.smallrye-health.check.\"io.quarkus.kafka.client.health.KafkaHealthCheck\".enabled"
+ dataIndexServiceUrlProtocol = "http"
+ jobServiceURLProperty = "mp.messaging.outgoing.kogito-job-service-job-request-events.url"
+ jobServiceKafkaSinkInjectionHealthCheck = `quarkus.smallrye-health.check."org.kie.kogito.jobs.service.messaging.http.health.knative.KSinkInjectionHealthCheck".enabled`
+ jobServiceStatusChangeEventsProperty = "kogito.jobs-service.http.job-status-change-events"
+ jobServiceStatusChangeEventsURL = "mp.messaging.outgoing.kogito-job-service-job-status-events-http.url"
+ jobServiceURLProtocol = "http"
+ jobServiceDataSourceReactiveURLProperty = "quarkus.datasource.reactive.url"
+
+ imageNamePrefix = "quay.io/kiegroup/kogito"
+ DataIndexServiceName = "data-index-service"
+ DataIndexName = "data-index"
+ JobServiceName = "jobs-service"
+ PersistenceTypeEphemeral = "ephemeral"
+ PersistenceTypePostgressql = "postgresql"
microprofileServiceCatalogPropertyPrefix = "org.kie.kogito.addons.discovery."
discoveryLikePropertyPattern = "^\\${(kubernetes|knative|openshift):(.*)}$"
+
+ defaultDatabaseName string = "sonataflow"
+ defaultPostgreSQLPort int = 5432
+)
+
+var (
+ immutableApplicationProperties = "quarkus.http.port=" + DefaultHTTPWorkflowPortIntStr.String() + "\n" +
+ "quarkus.http.host=0.0.0.0\n" +
+ // We disable the Knative health checks to not block the dev pod to run if Knative objects are not available
+ // See: https://kiegroup.github.io/kogito-docs/serverlessworkflow/latest/eventing/consume-produce-events-with-knative-eventing.html#ref-knative-eventing-add-on-source-configuration
+ "org.kie.kogito.addons.knative.eventing.health-enabled=false\n" +
+ "quarkus.devservices.enabled=false\n" +
+ "quarkus.kogito.devservices.enabled=false\n"
+
+ discoveryLikePropertyExpr = regexp.MustCompile(discoveryLikePropertyPattern)
+ _ AppPropertyHandler = &appPropertyHandler{}
)
-var immutableApplicationProperties = "quarkus.http.port=" + DefaultHTTPWorkflowPortIntStr.String() + "\n" +
- "quarkus.http.host=0.0.0.0\n" +
- // We disable the Knative health checks to not block the dev pod to run if Knative objects are not available
- // See: https://kiegroup.github.io/kogito-docs/serverlessworkflow/latest/eventing/consume-produce-events-with-knative-eventing.html#ref-knative-eventing-add-on-source-configuration
- "org.kie.kogito.addons.knative.eventing.health-enabled=false\n" +
- "quarkus.devservices.enabled=false\n" +
- "quarkus.kogito.devservices.enabled=false\n"
+type PlatformService interface {
Review Comment:
Moved inside the`platform` package.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "ricardozanini (via GitHub)" <gi...@apache.org>.
ricardozanini commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1419409465
##########
controllers/platform/services/properties.go:
##########
@@ -0,0 +1,97 @@
+// Copyright 2023 Apache Software Foundation (ASF)
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package services
+
+import (
+ "fmt"
+ "strings"
+
+ operatorapi "github.com/apache/incubator-kie-kogito-serverless-operator/api/v1alpha08"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles/common/constants"
+
+ "github.com/magiconair/properties"
+)
+
+func generateReactiveURL(postgresSpec *operatorapi.PersistencePostgreSql, schema string, namespace string, dbName string, port int) string {
+ if len(postgresSpec.JdbcUrl) > 0 && strings.Contains(postgresSpec.JdbcUrl, "currentSchema=") {
+ return strings.Replace(strings.TrimPrefix(postgresSpec.JdbcUrl, "jdbc:"), "currentSchema=", "search_path=", -1)
+ }
+ databaseSchema := schema
+ if len(postgresSpec.ServiceRef.DatabaseSchema) > 0 {
+ databaseSchema = postgresSpec.ServiceRef.DatabaseSchema
+ }
+ databaseNamespace := namespace
+ if len(postgresSpec.ServiceRef.Namespace) > 0 {
+ databaseNamespace = postgresSpec.ServiceRef.Namespace
+ }
+ dataSourcePort := port
+ if postgresSpec.ServiceRef.Port != nil {
+ dataSourcePort = *postgresSpec.ServiceRef.Port
+ }
+ databaseName := dbName
+ if len(postgresSpec.ServiceRef.DatabaseName) > 0 {
+ databaseName = postgresSpec.ServiceRef.DatabaseName
+ }
+ return fmt.Sprintf("%s://%s:%d/%s?search_path=%s", constants.PersistenceTypePostgreSQL, postgresSpec.ServiceRef.Name+"."+databaseNamespace, dataSourcePort, databaseName, databaseSchema)
+}
+
+// withDataIndexServiceUrl adds the property dataIndexServiceUrlProperty to the application properties.
+// See Service Discovery https://kubernetes.io/docs/concepts/services-networking/service/#dns
+func GenerateDataIndexApplicationProperties(workflow *operatorapi.SonataFlow, platform *operatorapi.SonataFlowPlatform) *properties.Properties {
Review Comment:
I think this comment is displaced now.
##########
controllers/platform/services/properties.go:
##########
@@ -0,0 +1,97 @@
+// Copyright 2023 Apache Software Foundation (ASF)
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package services
+
+import (
+ "fmt"
+ "strings"
+
+ operatorapi "github.com/apache/incubator-kie-kogito-serverless-operator/api/v1alpha08"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles/common/constants"
+
+ "github.com/magiconair/properties"
+)
+
+func generateReactiveURL(postgresSpec *operatorapi.PersistencePostgreSql, schema string, namespace string, dbName string, port int) string {
+ if len(postgresSpec.JdbcUrl) > 0 && strings.Contains(postgresSpec.JdbcUrl, "currentSchema=") {
+ return strings.Replace(strings.TrimPrefix(postgresSpec.JdbcUrl, "jdbc:"), "currentSchema=", "search_path=", -1)
+ }
+ databaseSchema := schema
+ if len(postgresSpec.ServiceRef.DatabaseSchema) > 0 {
+ databaseSchema = postgresSpec.ServiceRef.DatabaseSchema
+ }
+ databaseNamespace := namespace
+ if len(postgresSpec.ServiceRef.Namespace) > 0 {
+ databaseNamespace = postgresSpec.ServiceRef.Namespace
+ }
+ dataSourcePort := port
+ if postgresSpec.ServiceRef.Port != nil {
+ dataSourcePort = *postgresSpec.ServiceRef.Port
+ }
+ databaseName := dbName
+ if len(postgresSpec.ServiceRef.DatabaseName) > 0 {
+ databaseName = postgresSpec.ServiceRef.DatabaseName
+ }
+ return fmt.Sprintf("%s://%s:%d/%s?search_path=%s", constants.PersistenceTypePostgreSQL, postgresSpec.ServiceRef.Name+"."+databaseNamespace, dataSourcePort, databaseName, databaseSchema)
+}
+
+// withDataIndexServiceUrl adds the property dataIndexServiceUrlProperty to the application properties.
+// See Service Discovery https://kubernetes.io/docs/concepts/services-networking/service/#dns
+func GenerateDataIndexApplicationProperties(workflow *operatorapi.SonataFlow, platform *operatorapi.SonataFlowPlatform) *properties.Properties {
+ props := properties.NewProperties()
+ if profiles.IsProdProfile(workflow) && dataIndexEnabled(platform) {
+ di := NewDataIndexService(platform)
+ props.Set(
+ constants.DataIndexServiceURLProperty,
+ fmt.Sprintf("%s://%s.%s/processes", constants.DataIndexServiceURLProtocol, di.GetServiceName(), platform.Namespace),
+ )
+ }
+ return props
+}
+
+// withJobServiceURL adds the property 'mp.messaging.outgoing.kogito-job-service-job-request-events.url' to the application property
+func GenerateJobServiceApplicationProperties(workflow *operatorapi.SonataFlow, platform *operatorapi.SonataFlowPlatform) *properties.Properties {
Review Comment:
Same here regarding the comments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "ricardozanini (via GitHub)" <gi...@apache.org>.
ricardozanini commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1422796111
##########
controllers/platform/services.go:
##########
@@ -58,30 +55,32 @@ func (action *serviceAction) Handle(ctx context.Context, platform *operatorapi.S
return nil, err
}
- if err := createDataIndexComponents(ctx, action.client, platform); err != nil {
- return nil, err
+ if platform.Spec.Services.DataIndex != nil {
Review Comment:
Fair, I remembered now sorry for the noise. @masayag can you open an issue here to implement this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "tchughesiv (via GitHub)" <gi...@apache.org>.
tchughesiv commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1422748685
##########
controllers/platform/services.go:
##########
@@ -58,30 +55,32 @@ func (action *serviceAction) Handle(ctx context.Context, platform *operatorapi.S
return nil, err
}
- if err := createDataIndexComponents(ctx, action.client, platform); err != nil {
- return nil, err
+ if platform.Spec.Services.DataIndex != nil {
Review Comment:
I also think this all sounds reasonable, with `RollingUpdate` as the default when this new "flag" is not set. @ricardozanini I'm not sure the cluster-scoped changes would need to wait for this though... what is your thinking with that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1424403721
##########
controllers/platform/services/services.go:
##########
@@ -0,0 +1,298 @@
+// Copyright 2023 Red Hat, Inc. and/or its affiliates
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package services
+
+import (
+ "fmt"
+ "strconv"
+
+ corev1 "k8s.io/api/core/v1"
+ "k8s.io/apimachinery/pkg/api/resource"
+
+ operatorapi "github.com/apache/incubator-kie-kogito-serverless-operator/api/v1alpha08"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles/common/constants"
+
+ "github.com/apache/incubator-kie-kogito-serverless-operator/version"
+ "github.com/imdario/mergo"
+)
+
+type Platform interface {
+ // GetContainerName returns the name of the service's container in the deployment.
+ GetContainerName() string
+ // GetServiceImageName returns the image name of the service's container. It takes in the service and persistence types and returns a string
+ // that contains the FQDN of the image, including the tag.
+ GetServiceImageName(persistenceName string) string
+ // GetServiceName returns the name of the kubernetes service prefixed with the platform name
+ GetServiceName() string
+ // GetServiceCmName returns the name of the configmap associated to the service
+ GetServiceCmName() string
+ // GetEnvironmentVariables returns the env variables to be injected to the service container
+ GetEnvironmentVariables() []corev1.EnvVar
+ // GetResourceLimits returns the pod's memory and CPU resource requirements
+ // Values for job service taken from
+ // https://github.com/parodos-dev/orchestrator-helm-chart/blob/52d09eda56fdbed3060782df29847c97f172600f/charts/orchestrator/values.yaml#L68-L72
+ GetPodResourceRequirements() corev1.ResourceRequirements
+ // GetReplicaCountForService Returns the default pod replica count for the given service
+ GetReplicaCount() int32
Review Comment:
ok so it should always set to 1 regardless of what it's elswhere defined, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1424842590
##########
controllers/platform/services/services.go:
##########
@@ -0,0 +1,298 @@
+// Copyright 2023 Red Hat, Inc. and/or its affiliates
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package services
+
+import (
+ "fmt"
+ "strconv"
+
+ corev1 "k8s.io/api/core/v1"
+ "k8s.io/apimachinery/pkg/api/resource"
+
+ operatorapi "github.com/apache/incubator-kie-kogito-serverless-operator/api/v1alpha08"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles/common/constants"
+
+ "github.com/apache/incubator-kie-kogito-serverless-operator/version"
+ "github.com/imdario/mergo"
+)
+
+type Platform interface {
+ // GetContainerName returns the name of the service's container in the deployment.
+ GetContainerName() string
+ // GetServiceImageName returns the image name of the service's container. It takes in the service and persistence types and returns a string
+ // that contains the FQDN of the image, including the tag.
+ GetServiceImageName(persistenceName string) string
+ // GetServiceName returns the name of the kubernetes service prefixed with the platform name
+ GetServiceName() string
+ // GetServiceCmName returns the name of the configmap associated to the service
+ GetServiceCmName() string
+ // GetEnvironmentVariables returns the env variables to be injected to the service container
+ GetEnvironmentVariables() []corev1.EnvVar
+ // GetResourceLimits returns the pod's memory and CPU resource requirements
+ // Values for job service taken from
+ // https://github.com/parodos-dev/orchestrator-helm-chart/blob/52d09eda56fdbed3060782df29847c97f172600f/charts/orchestrator/values.yaml#L68-L72
+ GetPodResourceRequirements() corev1.ResourceRequirements
+ // GetReplicaCountForService Returns the default pod replica count for the given service
+ GetReplicaCount() int32
+
+ // MergeContainerSpec performs a merge with override using the containerSpec argument and the expected values based on the service's pod template specifications. The returning
+ // object is the merged result
+ MergeContainerSpec(containerSpec *corev1.Container) (*corev1.Container, error)
+
+ //ConfigurePersistence sets the persistence's image and environment values when it is defined in the Persistence field of the service, overriding any existing value.
+ ConfigurePersistence(containerSpec *corev1.Container) *corev1.Container
+
+ //MergePodSpec performs a merge with override between the podSpec argument and the expected values based on the service's pod template specification. The returning
+ // object is the result of the merge
+ MergePodSpec(podSpec corev1.PodSpec) (corev1.PodSpec, error)
+}
+
+type DataIndex struct {
+ platform *operatorapi.SonataFlowPlatform
+}
+
+func NewDataIndexService(platform *operatorapi.SonataFlowPlatform) DataIndex {
+ return DataIndex{platform: platform}
+}
+
+func (d DataIndex) GetContainerName() string {
+ return constants.DataIndexServiceName
+}
+
+func (d DataIndex) GetServiceImageName(persistenceName string) string {
+ var tag = version.GetMajorMinor()
+ if version.IsSnapshot() {
+ tag = "latest"
+ }
+ // returns "quay.io/kiegroup/kogito-data-index-<persistence_layer>:<tag>"
+ return fmt.Sprintf("%s-%s-%s:%s", constants.ImageNamePrefix, constants.DataIndexName, persistenceName, tag)
+}
+
+func (d DataIndex) GetServiceName() string {
+ return fmt.Sprintf("%s-%s", d.platform.Name, constants.DataIndexServiceName)
+}
+
+func (d DataIndex) GetEnvironmentVariables() []corev1.EnvVar {
+ return []corev1.EnvVar{
+ {
+ Name: "KOGITO_DATA_INDEX_QUARKUS_PROFILE",
+ Value: "http-events-support",
+ },
+ {
+ Name: "QUARKUS_HTTP_CORS",
+ Value: "true",
+ },
+ {
+ Name: "QUARKUS_HTTP_CORS_ORIGINS",
+ Value: "/.*/",
+ },
+ }
+}
+
+func (d DataIndex) GetPodResourceRequirements() corev1.ResourceRequirements {
+ return corev1.ResourceRequirements{
+ Limits: corev1.ResourceList{
+ corev1.ResourceCPU: resource.MustParse("100m"),
+ corev1.ResourceMemory: resource.MustParse("256Mi"),
+ },
+ }
+}
+
+func (d DataIndex) MergePodSpec(podSpec corev1.PodSpec) (corev1.PodSpec, error) {
+ c := podSpec.DeepCopy()
+ err := mergo.Merge(c, d.platform.Spec.Services.DataIndex.PodTemplate.PodSpec.ToPodSpec(), mergo.WithOverride)
+ return *c, err
+}
+
+func (d DataIndex) ConfigurePersistence(containerSpec *corev1.Container) *corev1.Container {
+
+ if d.platform.Spec.Services.DataIndex.Persistence != nil && d.platform.Spec.Services.DataIndex.Persistence.PostgreSql != nil {
+ c := containerSpec.DeepCopy()
+ c.Image = d.GetServiceImageName(constants.PersistenceTypePostgreSQL)
+ c.Env = append(c.Env, configurePostgreSqlEnv(d.platform.Spec.Services.DataIndex.Persistence.PostgreSql, d.GetServiceName(), d.platform.Namespace)...)
+ return c
+ }
+ return containerSpec
+}
+
+func (d DataIndex) MergeContainerSpec(containerSpec *corev1.Container) (*corev1.Container, error) {
+ c := containerSpec.DeepCopy()
+ err := mergo.Merge(c, d.platform.Spec.Services.DataIndex.PodTemplate.Container.ToContainer(), mergo.WithOverride)
+ return c, err
+}
+
+func (d DataIndex) GetReplicaCount() int32 {
+ if d.platform.Spec.Services.DataIndex.PodTemplate.Replicas != nil {
+ return *d.platform.Spec.Services.DataIndex.PodTemplate.Replicas
+ }
+ return 1
+}
+
+func (d DataIndex) GetServiceCmName() string {
+ return fmt.Sprintf("%s-props", d.GetServiceName())
+}
+
+type JobService struct {
+ platform *operatorapi.SonataFlowPlatform
+}
+
+func NewJobService(platform *operatorapi.SonataFlowPlatform) JobService {
+ return JobService{platform: platform}
+}
+
+func (j JobService) GetContainerName() string {
+ return constants.JobServiceName
+}
+
+func (j JobService) GetServiceImageName(persistenceName string) string {
+ var tag = version.GetMajorMinor()
+ if version.IsSnapshot() {
+ tag = "latest"
+ }
+ // returns "quay.io/kiegroup/kogito-jobs-service-<persistece_layer>:<tag>"
+ return fmt.Sprintf("%s-%s-%s:%s", constants.ImageNamePrefix, constants.JobServiceName, persistenceName, tag)
+}
+
+func (j JobService) GetServiceName() string {
+ return fmt.Sprintf("%s-%s", j.platform.Name, constants.JobServiceName)
+}
+
+func (j JobService) GetServiceCmName() string {
+ return fmt.Sprintf("%s-props", j.GetServiceName())
+}
+
+func (j JobService) GetEnvironmentVariables() []corev1.EnvVar {
+ return []corev1.EnvVar{
+ {
+ Name: "QUARKUS_HTTP_CORS",
+ Value: "true",
+ },
+ {
+ Name: "QUARKUS_HTTP_CORS_ORIGINS",
+ Value: "/.*/",
+ },
+ }
+}
+
+func (j JobService) GetPodResourceRequirements() corev1.ResourceRequirements {
+ return corev1.ResourceRequirements{
+ Requests: corev1.ResourceList{
+ corev1.ResourceCPU: resource.MustParse("250m"),
+ corev1.ResourceMemory: resource.MustParse("64Mi"),
+ },
+ Limits: corev1.ResourceList{
+ corev1.ResourceCPU: resource.MustParse("500m"),
+ corev1.ResourceMemory: resource.MustParse("1Gi"),
+ },
+ }
+}
+
+func (j JobService) GetReplicaCount() int32 {
+ if j.platform.Spec.Services.JobService.PodTemplate.Replicas != nil {
+ return *j.platform.Spec.Services.JobService.PodTemplate.Replicas
+ }
+ return 1
+}
+
+func (j JobService) MergeContainerSpec(containerSpec *corev1.Container) (*corev1.Container, error) {
+ c := containerSpec.DeepCopy()
+ err := mergo.Merge(c, j.platform.Spec.Services.JobService.PodTemplate.Container.ToContainer(), mergo.WithOverride)
+ return c, err
+}
+
+func (j JobService) ConfigurePersistence(containerSpec *corev1.Container) *corev1.Container {
+
+ if j.platform.Spec.Services.JobService.Persistence != nil && j.platform.Spec.Services.JobService.Persistence.PostgreSql != nil {
+ c := containerSpec.DeepCopy()
+ c.Image = j.GetServiceImageName(constants.PersistenceTypePostgreSQL)
+ c.Env = append(c.Env, configurePostgreSqlEnv(j.platform.Spec.Services.JobService.Persistence.PostgreSql, j.GetServiceName(), j.platform.Namespace)...)
+ return c
+ }
+ return containerSpec
+}
+
+func (j JobService) MergePodSpec(podSpec corev1.PodSpec) (corev1.PodSpec, error) {
+ c := podSpec.DeepCopy()
+ err := mergo.Merge(c, j.platform.Spec.Services.JobService.PodTemplate.PodSpec.ToPodSpec(), mergo.WithOverride)
+ return *c, err
+}
+
+func configurePostgreSqlEnv(postgresql *operatorapi.PersistencePostgreSql, databaseSchema, databaseNamespace string) []corev1.EnvVar {
+ if len(postgresql.ServiceRef.DatabaseSchema) > 0 {
+ databaseSchema = postgresql.ServiceRef.DatabaseSchema
+ }
+ if len(postgresql.ServiceRef.Namespace) > 0 {
+ databaseNamespace = postgresql.ServiceRef.Namespace
+ }
+ dataSourcePort := 5432
+ if postgresql.ServiceRef.Port != nil {
+ dataSourcePort = *postgresql.ServiceRef.Port
+ }
+ databaseName := "sonataflow"
+ if len(postgresql.ServiceRef.DatabaseName) > 0 {
+ databaseName = postgresql.ServiceRef.DatabaseName
+ }
+ dataSourceUrl := "jdbc:" + constants.PersistenceTypePostgreSQL + "://" + postgresql.ServiceRef.Name + "." + databaseNamespace + ":" + strconv.Itoa(dataSourcePort) + "/" + databaseName + "?currentSchema=" + databaseSchema
+ if len(postgresql.JdbcUrl) > 0 {
+ dataSourceUrl = postgresql.JdbcUrl
+ }
+ secretRef := corev1.LocalObjectReference{
+ Name: postgresql.SecretRef.Name,
+ }
+ quarkusDatasourceUsername := "POSTGRESQL_USER"
+ if len(postgresql.SecretRef.UserKey) > 0 {
+ quarkusDatasourceUsername = postgresql.SecretRef.UserKey
+ }
+ quarkusDatasourcePassword := "POSTGRESQL_PASSWORD"
+ if len(postgresql.SecretRef.PasswordKey) > 0 {
+ quarkusDatasourcePassword = postgresql.SecretRef.PasswordKey
+ }
+ return []corev1.EnvVar{
+ {
+ Name: "QUARKUS_DATASOURCE_USERNAME",
+ ValueFrom: &corev1.EnvVarSource{
+ SecretKeyRef: &corev1.SecretKeySelector{
+ Key: quarkusDatasourceUsername,
+ LocalObjectReference: secretRef,
+ },
+ },
+ },
+ {
+ Name: "QUARKUS_DATASOURCE_PASSWORD",
+ ValueFrom: &corev1.EnvVarSource{
+ SecretKeyRef: &corev1.SecretKeySelector{
+ Key: quarkusDatasourcePassword,
+ LocalObjectReference: secretRef,
+ },
+ },
+ },
+ {
+ Name: "QUARKUS_DATASOURCE_DB_KIND",
+ Value: constants.PersistenceTypePostgreSQL,
+ },
+ {
+ Name: "QUARKUS_HIBERNATE_ORM_DATABASE_GENERATION",
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1424842808
##########
controllers/platform/services/properties.go:
##########
@@ -0,0 +1,98 @@
+// Copyright 2023 Apache Software Foundation (ASF)
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package services
+
+import (
+ "fmt"
+ "strings"
+
+ operatorapi "github.com/apache/incubator-kie-kogito-serverless-operator/api/v1alpha08"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles/common/constants"
+
+ "github.com/magiconair/properties"
+)
+
+func generateReactiveURL(postgresSpec *operatorapi.PersistencePostgreSql, schema string, namespace string, dbName string, port int) string {
+ if len(postgresSpec.JdbcUrl) > 0 && strings.Contains(postgresSpec.JdbcUrl, "currentSchema=") {
+ return strings.Replace(strings.TrimPrefix(postgresSpec.JdbcUrl, "jdbc:"), "currentSchema=", "search_path=", -1)
+ }
+ databaseSchema := schema
+ if len(postgresSpec.ServiceRef.DatabaseSchema) > 0 {
+ databaseSchema = postgresSpec.ServiceRef.DatabaseSchema
+ }
+ databaseNamespace := namespace
+ if len(postgresSpec.ServiceRef.Namespace) > 0 {
+ databaseNamespace = postgresSpec.ServiceRef.Namespace
+ }
+ dataSourcePort := port
+ if postgresSpec.ServiceRef.Port != nil {
+ dataSourcePort = *postgresSpec.ServiceRef.Port
+ }
+ databaseName := dbName
+ if len(postgresSpec.ServiceRef.DatabaseName) > 0 {
+ databaseName = postgresSpec.ServiceRef.DatabaseName
+ }
+ return fmt.Sprintf("%s://%s:%d/%s?search_path=%s", constants.PersistenceTypePostgreSQL, postgresSpec.ServiceRef.Name+"."+databaseNamespace, dataSourcePort, databaseName, databaseSchema)
+}
+
+// GenerateDataIndexApplicationProperties returns the application properties required for the Data Index Service to work when deployed in a production profile
+// and data index' service's spec field `Enabled` set to true
+func GenerateDataIndexApplicationProperties(workflow *operatorapi.SonataFlow, platform *operatorapi.SonataFlowPlatform) *properties.Properties {
+ props := properties.NewProperties()
+ if profiles.IsProdProfile(workflow) && dataIndexEnabled(platform) {
+ di := NewDataIndexService(platform)
+ props.Set(
+ constants.DataIndexServiceURLProperty,
+ fmt.Sprintf("%s://%s.%s/processes", constants.DataIndexServiceURLProtocol, di.GetServiceName(), platform.Namespace),
+ )
+ }
+ return props
+}
+
+// GenerateJobServiceApplicationProperties returns the application properties required for the Job Service to work in a production profile and job service's
+// service's spec field `Enabled` set to true
+func GenerateJobServiceApplicationProperties(workflow *operatorapi.SonataFlow, platform *operatorapi.SonataFlowPlatform) *properties.Properties {
+ props := properties.NewProperties()
+ if profiles.IsProdProfile(workflow) && jobServiceEnabled(platform) {
Review Comment:
Removed references to the workflow in both data index and job service application properties generation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1426125088
##########
controllers/platform/services/services.go:
##########
@@ -0,0 +1,298 @@
+// Copyright 2023 Red Hat, Inc. and/or its affiliates
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package services
+
+import (
+ "fmt"
+ "strconv"
+
+ corev1 "k8s.io/api/core/v1"
+ "k8s.io/apimachinery/pkg/api/resource"
+
+ operatorapi "github.com/apache/incubator-kie-kogito-serverless-operator/api/v1alpha08"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles/common/constants"
+
+ "github.com/apache/incubator-kie-kogito-serverless-operator/version"
+ "github.com/imdario/mergo"
+)
+
+type Platform interface {
+ // GetContainerName returns the name of the service's container in the deployment.
+ GetContainerName() string
+ // GetServiceImageName returns the image name of the service's container. It takes in the service and persistence types and returns a string
+ // that contains the FQDN of the image, including the tag.
+ GetServiceImageName(persistenceName string) string
+ // GetServiceName returns the name of the kubernetes service prefixed with the platform name
+ GetServiceName() string
+ // GetServiceCmName returns the name of the configmap associated to the service
+ GetServiceCmName() string
+ // GetEnvironmentVariables returns the env variables to be injected to the service container
+ GetEnvironmentVariables() []corev1.EnvVar
+ // GetResourceLimits returns the pod's memory and CPU resource requirements
+ // Values for job service taken from
+ // https://github.com/parodos-dev/orchestrator-helm-chart/blob/52d09eda56fdbed3060782df29847c97f172600f/charts/orchestrator/values.yaml#L68-L72
+ GetPodResourceRequirements() corev1.ResourceRequirements
+ // GetReplicaCountForService Returns the default pod replica count for the given service
+ GetReplicaCount() int32
Review Comment:
Resolving this comment as it has already answered earlier
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "ricardozanini (via GitHub)" <gi...@apache.org>.
ricardozanini commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1860588248
@jordigilh the e2e ` should successfully deploy the orderprocessing workflow in devmode and verify if it's running` is failing. You can run these from your local environment using `./local/hack/run-e2e.sh`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "wmedvede (via GitHub)" <gi...@apache.org>.
wmedvede commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1863203823
> ```java
> orderprocessing-5d586b99bb-vq7rp ERROR [io.quarkus.runtime.Application:-1] (Quarkus Main Thread) Failed to start application (with profile [dev]): java.lang.IllegalArgumentException: Unsupported scheme for the http connector in URL: services.v1/event-listener
> ```
Hi @jordigilh , this PR https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/332 should fix the issue regarding the E2E tests. No you'll find something like the screenshot blow in the WF properties, and the event-listener address is resolved well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1858127112
> Hi @jordigilh
>
> > in other words: the properties specific for creating the job service are all included in the properties for a sonataflow CR? or a subset of those?
>
> > Right now I consider them to be the same.
>
> No, they are not the same. I think it was there that things messed a little bit. Take a look at this repo we shared at the beginning of the development:
>
> https://github.com/flows-examples/techpreview2/tree/main/infra/dataindex_and_jobservice
>
> You can see the JS Deployment and the `application.properties` for it there. That's exactly what we were expecting for this PR to do.
>
> And then, if you see the use case: https://github.com/flows-examples/techpreview2/tree/main?tab=readme-ov-file#use-case-3
>
> It's explained there how it is supposed to be working. The workflow has yet another set of properties: https://github.com/flows-examples/techpreview2/blob/main/workflows/sonataflow-callbackstatetimeouts-persistence/02-configmap_callbackstatetimeouts-props
>
> That's why I asked to separate the props in the first place last week between the services and the workflow. I thought it was clear since these use cases were already working and defined in that repo. I'm sorry if I made an incorrect assumption.
Ok, so for the sake of clarity because I'm not yet sure what should be in where:
* When deploying the job service alone, the properties that should go specific for the job service are as follows (just adding the key here for simplicity):
* `kogito.service.url`
* `quarkus.datasource.jdbc.url`
* `quarkus.datasource.reactive.url`
* `kogito.jobs-service.http.job-status-change-events`
* `mp.messaging.outgoing.kogito-job-service-job-status-events-http.url`
* `quarkus.smallrye-health.check."org.kie.kogito.jobs.service.messaging.http.health.knative.KSinkInjectionHealthCheck".enabled`
* When Data Index is deployed with a nonDev profile and with the enabled field set to `true`, additional properties to be added:
* `kogito.jobs-service.http.job-status-change-events`
* `mp.messaging.outgoing.kogito-job-service-job-status-events-http.url`
* When deploying a workflow, these are the keys expected generated by the job service with a postgreSQL DB for persistence:
* `mp.messaging.outgoing.kogito-job-service-job-request-events.connector`
* `mp.messaging.outgoing.kogito-job-service-job-request-events.url`
I can't find any property shared between the job service deployment and a workflow deployment.
From the [application.properties](https://github.com/flows-examples/techpreview2/blob/main/workflows/sonataflow-callbackstatetimeouts-persistence/02-configmap_callbackstatetimeouts-props.yaml) in the example @ricardozanini provided, I can see only 3 of the 4 properties that @wmedvede mentioned earlier in the case of deployment of a workflow: one listed in the job service section and the other 2 outside of it, which would mean they're out of scope of this PR:` kogito.events.usertasks.enabled`, `kogito.events.variables.enabled`. Missing is `kogito.events.processdefinitions.enabled`.
Is this configuration correct?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "masayag (via GitHub)" <gi...@apache.org>.
masayag commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1422549174
##########
controllers/platform/services.go:
##########
@@ -58,30 +55,32 @@ func (action *serviceAction) Handle(ctx context.Context, platform *operatorapi.S
return nil, err
}
- if err := createDataIndexComponents(ctx, action.client, platform); err != nil {
- return nil, err
+ if platform.Spec.Services.DataIndex != nil {
Review Comment:
Maybe this was discussed before, so dropping some ideas here:
Perhaps the desired behavior here is more aligned with [k8s rolloutStrategy](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.26/#deploymentstrategy-v1-apps).
I'd explore the idea of replacing the `Enabled` flag with a `rolloutStrategy` property.
It leaves more room for introducing behavior types on CR change. In case of a change should be reflected in the workflows, the `rolloutStrategy` will determine the behavior. Few values to `rolloutStrategy`:
* Disabled - do nothing when platform CR is changed. this requires some out-of-sync between platform configuration to what is currently served by the workflows.
* Recreate - restart all with new configuration
* RollingUpdate - rolling the changes.
* Detached - detaching the service from workflow definition.
I'm also okay with not having it at all, but making the sync-status with the platform configuration visible is useful for debugging.
For once, I'd go with a single flag or strategy for all of the services and not per service.
Also, perhaps the future will knative service will not need this type of handling at all.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "wmedvede (via GitHub)" <gi...@apache.org>.
wmedvede commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1424318772
##########
controllers/platform/services/properties.go:
##########
@@ -0,0 +1,98 @@
+// Copyright 2023 Apache Software Foundation (ASF)
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package services
+
+import (
+ "fmt"
+ "strings"
+
+ operatorapi "github.com/apache/incubator-kie-kogito-serverless-operator/api/v1alpha08"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles/common/constants"
+
+ "github.com/magiconair/properties"
+)
+
+func generateReactiveURL(postgresSpec *operatorapi.PersistencePostgreSql, schema string, namespace string, dbName string, port int) string {
+ if len(postgresSpec.JdbcUrl) > 0 && strings.Contains(postgresSpec.JdbcUrl, "currentSchema=") {
+ return strings.Replace(strings.TrimPrefix(postgresSpec.JdbcUrl, "jdbc:"), "currentSchema=", "search_path=", -1)
+ }
+ databaseSchema := schema
+ if len(postgresSpec.ServiceRef.DatabaseSchema) > 0 {
+ databaseSchema = postgresSpec.ServiceRef.DatabaseSchema
+ }
+ databaseNamespace := namespace
+ if len(postgresSpec.ServiceRef.Namespace) > 0 {
+ databaseNamespace = postgresSpec.ServiceRef.Namespace
+ }
+ dataSourcePort := port
+ if postgresSpec.ServiceRef.Port != nil {
+ dataSourcePort = *postgresSpec.ServiceRef.Port
+ }
+ databaseName := dbName
+ if len(postgresSpec.ServiceRef.DatabaseName) > 0 {
+ databaseName = postgresSpec.ServiceRef.DatabaseName
+ }
+ return fmt.Sprintf("%s://%s:%d/%s?search_path=%s", constants.PersistenceTypePostgreSQL, postgresSpec.ServiceRef.Name+"."+databaseNamespace, dataSourcePort, databaseName, databaseSchema)
+}
+
+// GenerateDataIndexApplicationProperties returns the application properties required for the Data Index Service to work when deployed in a production profile
+// and data index' service's spec field `Enabled` set to true
+func GenerateDataIndexApplicationProperties(workflow *operatorapi.SonataFlow, platform *operatorapi.SonataFlowPlatform) *properties.Properties {
+ props := properties.NewProperties()
+ if profiles.IsProdProfile(workflow) && dataIndexEnabled(platform) {
+ di := NewDataIndexService(platform)
+ props.Set(
+ constants.DataIndexServiceURLProperty,
+ fmt.Sprintf("%s://%s.%s/processes", constants.DataIndexServiceURLProtocol, di.GetServiceName(), platform.Namespace),
+ )
+ }
+ return props
+}
+
+// GenerateJobServiceApplicationProperties returns the application properties required for the Job Service to work in a production profile and job service's
+// service's spec field `Enabled` set to true
+func GenerateJobServiceApplicationProperties(workflow *operatorapi.SonataFlow, platform *operatorapi.SonataFlowPlatform) *properties.Properties {
+ props := properties.NewProperties()
+ if profiles.IsProdProfile(workflow) && jobServiceEnabled(platform) {
Review Comment:
btw, @jordigilh @ricardozanini the comment above regarding the IsProdProfile check apply also for the data-index
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "wmedvede (via GitHub)" <gi...@apache.org>.
wmedvede commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1424284261
##########
controllers/platform/services/properties.go:
##########
@@ -0,0 +1,98 @@
+// Copyright 2023 Apache Software Foundation (ASF)
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package services
+
+import (
+ "fmt"
+ "strings"
+
+ operatorapi "github.com/apache/incubator-kie-kogito-serverless-operator/api/v1alpha08"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles/common/constants"
+
+ "github.com/magiconair/properties"
+)
+
+func generateReactiveURL(postgresSpec *operatorapi.PersistencePostgreSql, schema string, namespace string, dbName string, port int) string {
+ if len(postgresSpec.JdbcUrl) > 0 && strings.Contains(postgresSpec.JdbcUrl, "currentSchema=") {
+ return strings.Replace(strings.TrimPrefix(postgresSpec.JdbcUrl, "jdbc:"), "currentSchema=", "search_path=", -1)
+ }
+ databaseSchema := schema
+ if len(postgresSpec.ServiceRef.DatabaseSchema) > 0 {
+ databaseSchema = postgresSpec.ServiceRef.DatabaseSchema
+ }
+ databaseNamespace := namespace
+ if len(postgresSpec.ServiceRef.Namespace) > 0 {
+ databaseNamespace = postgresSpec.ServiceRef.Namespace
+ }
+ dataSourcePort := port
+ if postgresSpec.ServiceRef.Port != nil {
+ dataSourcePort = *postgresSpec.ServiceRef.Port
+ }
+ databaseName := dbName
+ if len(postgresSpec.ServiceRef.DatabaseName) > 0 {
+ databaseName = postgresSpec.ServiceRef.DatabaseName
+ }
+ return fmt.Sprintf("%s://%s:%d/%s?search_path=%s", constants.PersistenceTypePostgreSQL, postgresSpec.ServiceRef.Name+"."+databaseNamespace, dataSourcePort, databaseName, databaseSchema)
+}
+
+// GenerateDataIndexApplicationProperties returns the application properties required for the Data Index Service to work when deployed in a production profile
+// and data index' service's spec field `Enabled` set to true
+func GenerateDataIndexApplicationProperties(workflow *operatorapi.SonataFlow, platform *operatorapi.SonataFlowPlatform) *properties.Properties {
+ props := properties.NewProperties()
+ if profiles.IsProdProfile(workflow) && dataIndexEnabled(platform) {
+ di := NewDataIndexService(platform)
+ props.Set(
+ constants.DataIndexServiceURLProperty,
+ fmt.Sprintf("%s://%s.%s/processes", constants.DataIndexServiceURLProtocol, di.GetServiceName(), platform.Namespace),
+ )
+ }
+ return props
+}
+
+// GenerateJobServiceApplicationProperties returns the application properties required for the Job Service to work in a production profile and job service's
+// service's spec field `Enabled` set to true
+func GenerateJobServiceApplicationProperties(workflow *operatorapi.SonataFlow, platform *operatorapi.SonataFlowPlatform) *properties.Properties {
+ props := properties.NewProperties()
+ if profiles.IsProdProfile(workflow) && jobServiceEnabled(platform) {
Review Comment:
maybe this condition explains what we chatted, note that if no workflow is present and we create a platform with the jobs-service it's created.
##########
controllers/platform/services/services.go:
##########
@@ -0,0 +1,298 @@
+// Copyright 2023 Red Hat, Inc. and/or its affiliates
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package services
+
+import (
+ "fmt"
+ "strconv"
+
+ corev1 "k8s.io/api/core/v1"
+ "k8s.io/apimachinery/pkg/api/resource"
+
+ operatorapi "github.com/apache/incubator-kie-kogito-serverless-operator/api/v1alpha08"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles/common/constants"
+
+ "github.com/apache/incubator-kie-kogito-serverless-operator/version"
+ "github.com/imdario/mergo"
+)
+
+type Platform interface {
+ // GetContainerName returns the name of the service's container in the deployment.
+ GetContainerName() string
+ // GetServiceImageName returns the image name of the service's container. It takes in the service and persistence types and returns a string
+ // that contains the FQDN of the image, including the tag.
+ GetServiceImageName(persistenceName string) string
+ // GetServiceName returns the name of the kubernetes service prefixed with the platform name
+ GetServiceName() string
+ // GetServiceCmName returns the name of the configmap associated to the service
+ GetServiceCmName() string
+ // GetEnvironmentVariables returns the env variables to be injected to the service container
+ GetEnvironmentVariables() []corev1.EnvVar
+ // GetResourceLimits returns the pod's memory and CPU resource requirements
+ // Values for job service taken from
+ // https://github.com/parodos-dev/orchestrator-helm-chart/blob/52d09eda56fdbed3060782df29847c97f172600f/charts/orchestrator/values.yaml#L68-L72
+ GetPodResourceRequirements() corev1.ResourceRequirements
+ // GetReplicaCountForService Returns the default pod replica count for the given service
+ GetReplicaCount() int32
+
+ // MergeContainerSpec performs a merge with override using the containerSpec argument and the expected values based on the service's pod template specifications. The returning
+ // object is the merged result
+ MergeContainerSpec(containerSpec *corev1.Container) (*corev1.Container, error)
+
+ //ConfigurePersistence sets the persistence's image and environment values when it is defined in the Persistence field of the service, overriding any existing value.
+ ConfigurePersistence(containerSpec *corev1.Container) *corev1.Container
+
+ //MergePodSpec performs a merge with override between the podSpec argument and the expected values based on the service's pod template specification. The returning
+ // object is the result of the merge
+ MergePodSpec(podSpec corev1.PodSpec) (corev1.PodSpec, error)
+}
+
+type DataIndex struct {
+ platform *operatorapi.SonataFlowPlatform
+}
+
+func NewDataIndexService(platform *operatorapi.SonataFlowPlatform) DataIndex {
+ return DataIndex{platform: platform}
+}
+
+func (d DataIndex) GetContainerName() string {
+ return constants.DataIndexServiceName
+}
+
+func (d DataIndex) GetServiceImageName(persistenceName string) string {
+ var tag = version.GetMajorMinor()
+ if version.IsSnapshot() {
+ tag = "latest"
+ }
+ // returns "quay.io/kiegroup/kogito-data-index-<persistence_layer>:<tag>"
+ return fmt.Sprintf("%s-%s-%s:%s", constants.ImageNamePrefix, constants.DataIndexName, persistenceName, tag)
+}
+
+func (d DataIndex) GetServiceName() string {
+ return fmt.Sprintf("%s-%s", d.platform.Name, constants.DataIndexServiceName)
+}
+
+func (d DataIndex) GetEnvironmentVariables() []corev1.EnvVar {
+ return []corev1.EnvVar{
+ {
+ Name: "KOGITO_DATA_INDEX_QUARKUS_PROFILE",
+ Value: "http-events-support",
+ },
+ {
+ Name: "QUARKUS_HTTP_CORS",
+ Value: "true",
+ },
+ {
+ Name: "QUARKUS_HTTP_CORS_ORIGINS",
+ Value: "/.*/",
+ },
+ }
+}
+
+func (d DataIndex) GetPodResourceRequirements() corev1.ResourceRequirements {
+ return corev1.ResourceRequirements{
+ Limits: corev1.ResourceList{
+ corev1.ResourceCPU: resource.MustParse("100m"),
+ corev1.ResourceMemory: resource.MustParse("256Mi"),
+ },
+ }
+}
+
+func (d DataIndex) MergePodSpec(podSpec corev1.PodSpec) (corev1.PodSpec, error) {
+ c := podSpec.DeepCopy()
+ err := mergo.Merge(c, d.platform.Spec.Services.DataIndex.PodTemplate.PodSpec.ToPodSpec(), mergo.WithOverride)
+ return *c, err
+}
+
+func (d DataIndex) ConfigurePersistence(containerSpec *corev1.Container) *corev1.Container {
+
+ if d.platform.Spec.Services.DataIndex.Persistence != nil && d.platform.Spec.Services.DataIndex.Persistence.PostgreSql != nil {
+ c := containerSpec.DeepCopy()
+ c.Image = d.GetServiceImageName(constants.PersistenceTypePostgreSQL)
+ c.Env = append(c.Env, configurePostgreSqlEnv(d.platform.Spec.Services.DataIndex.Persistence.PostgreSql, d.GetServiceName(), d.platform.Namespace)...)
+ return c
+ }
+ return containerSpec
+}
+
+func (d DataIndex) MergeContainerSpec(containerSpec *corev1.Container) (*corev1.Container, error) {
+ c := containerSpec.DeepCopy()
+ err := mergo.Merge(c, d.platform.Spec.Services.DataIndex.PodTemplate.Container.ToContainer(), mergo.WithOverride)
+ return c, err
+}
+
+func (d DataIndex) GetReplicaCount() int32 {
+ if d.platform.Spec.Services.DataIndex.PodTemplate.Replicas != nil {
+ return *d.platform.Spec.Services.DataIndex.PodTemplate.Replicas
+ }
+ return 1
+}
+
+func (d DataIndex) GetServiceCmName() string {
+ return fmt.Sprintf("%s-props", d.GetServiceName())
+}
+
+type JobService struct {
+ platform *operatorapi.SonataFlowPlatform
+}
+
+func NewJobService(platform *operatorapi.SonataFlowPlatform) JobService {
+ return JobService{platform: platform}
+}
+
+func (j JobService) GetContainerName() string {
+ return constants.JobServiceName
+}
+
+func (j JobService) GetServiceImageName(persistenceName string) string {
+ var tag = version.GetMajorMinor()
+ if version.IsSnapshot() {
+ tag = "latest"
+ }
+ // returns "quay.io/kiegroup/kogito-jobs-service-<persistece_layer>:<tag>"
+ return fmt.Sprintf("%s-%s-%s:%s", constants.ImageNamePrefix, constants.JobServiceName, persistenceName, tag)
+}
+
+func (j JobService) GetServiceName() string {
+ return fmt.Sprintf("%s-%s", j.platform.Name, constants.JobServiceName)
+}
+
+func (j JobService) GetServiceCmName() string {
+ return fmt.Sprintf("%s-props", j.GetServiceName())
+}
+
+func (j JobService) GetEnvironmentVariables() []corev1.EnvVar {
+ return []corev1.EnvVar{
+ {
+ Name: "QUARKUS_HTTP_CORS",
+ Value: "true",
+ },
+ {
+ Name: "QUARKUS_HTTP_CORS_ORIGINS",
+ Value: "/.*/",
+ },
+ }
+}
+
+func (j JobService) GetPodResourceRequirements() corev1.ResourceRequirements {
+ return corev1.ResourceRequirements{
+ Requests: corev1.ResourceList{
+ corev1.ResourceCPU: resource.MustParse("250m"),
+ corev1.ResourceMemory: resource.MustParse("64Mi"),
+ },
+ Limits: corev1.ResourceList{
+ corev1.ResourceCPU: resource.MustParse("500m"),
+ corev1.ResourceMemory: resource.MustParse("1Gi"),
+ },
+ }
+}
+
+func (j JobService) GetReplicaCount() int32 {
+ if j.platform.Spec.Services.JobService.PodTemplate.Replicas != nil {
+ return *j.platform.Spec.Services.JobService.PodTemplate.Replicas
+ }
+ return 1
+}
+
+func (j JobService) MergeContainerSpec(containerSpec *corev1.Container) (*corev1.Container, error) {
+ c := containerSpec.DeepCopy()
+ err := mergo.Merge(c, j.platform.Spec.Services.JobService.PodTemplate.Container.ToContainer(), mergo.WithOverride)
+ return c, err
+}
+
+func (j JobService) ConfigurePersistence(containerSpec *corev1.Container) *corev1.Container {
+
+ if j.platform.Spec.Services.JobService.Persistence != nil && j.platform.Spec.Services.JobService.Persistence.PostgreSql != nil {
+ c := containerSpec.DeepCopy()
+ c.Image = j.GetServiceImageName(constants.PersistenceTypePostgreSQL)
+ c.Env = append(c.Env, configurePostgreSqlEnv(j.platform.Spec.Services.JobService.Persistence.PostgreSql, j.GetServiceName(), j.platform.Namespace)...)
+ return c
+ }
+ return containerSpec
+}
+
+func (j JobService) MergePodSpec(podSpec corev1.PodSpec) (corev1.PodSpec, error) {
+ c := podSpec.DeepCopy()
+ err := mergo.Merge(c, j.platform.Spec.Services.JobService.PodTemplate.PodSpec.ToPodSpec(), mergo.WithOverride)
+ return *c, err
+}
+
+func configurePostgreSqlEnv(postgresql *operatorapi.PersistencePostgreSql, databaseSchema, databaseNamespace string) []corev1.EnvVar {
+ if len(postgresql.ServiceRef.DatabaseSchema) > 0 {
+ databaseSchema = postgresql.ServiceRef.DatabaseSchema
+ }
+ if len(postgresql.ServiceRef.Namespace) > 0 {
+ databaseNamespace = postgresql.ServiceRef.Namespace
+ }
+ dataSourcePort := 5432
+ if postgresql.ServiceRef.Port != nil {
+ dataSourcePort = *postgresql.ServiceRef.Port
+ }
+ databaseName := "sonataflow"
+ if len(postgresql.ServiceRef.DatabaseName) > 0 {
+ databaseName = postgresql.ServiceRef.DatabaseName
+ }
+ dataSourceUrl := "jdbc:" + constants.PersistenceTypePostgreSQL + "://" + postgresql.ServiceRef.Name + "." + databaseNamespace + ":" + strconv.Itoa(dataSourcePort) + "/" + databaseName + "?currentSchema=" + databaseSchema
+ if len(postgresql.JdbcUrl) > 0 {
+ dataSourceUrl = postgresql.JdbcUrl
+ }
+ secretRef := corev1.LocalObjectReference{
+ Name: postgresql.SecretRef.Name,
+ }
+ quarkusDatasourceUsername := "POSTGRESQL_USER"
+ if len(postgresql.SecretRef.UserKey) > 0 {
+ quarkusDatasourceUsername = postgresql.SecretRef.UserKey
+ }
+ quarkusDatasourcePassword := "POSTGRESQL_PASSWORD"
+ if len(postgresql.SecretRef.PasswordKey) > 0 {
+ quarkusDatasourcePassword = postgresql.SecretRef.PasswordKey
+ }
+ return []corev1.EnvVar{
+ {
+ Name: "QUARKUS_DATASOURCE_USERNAME",
+ ValueFrom: &corev1.EnvVarSource{
+ SecretKeyRef: &corev1.SecretKeySelector{
+ Key: quarkusDatasourceUsername,
+ LocalObjectReference: secretRef,
+ },
+ },
+ },
+ {
+ Name: "QUARKUS_DATASOURCE_PASSWORD",
+ ValueFrom: &corev1.EnvVarSource{
+ SecretKeyRef: &corev1.SecretKeySelector{
+ Key: quarkusDatasourcePassword,
+ LocalObjectReference: secretRef,
+ },
+ },
+ },
+ {
+ Name: "QUARKUS_DATASOURCE_DB_KIND",
+ Value: constants.PersistenceTypePostgreSQL,
+ },
+ {
+ Name: "QUARKUS_HIBERNATE_ORM_DATABASE_GENERATION",
Review Comment:
This env var don't apply for the jobs-service
##########
controllers/profiles/common/properties/application.go:
##########
@@ -27,46 +27,30 @@ import (
"strings"
"github.com/apache/incubator-kie-kogito-serverless-operator/controllers/discovery"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/platform/services"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles/common/constants"
"github.com/magiconair/properties"
"k8s.io/klog/v2"
operatorapi "github.com/apache/incubator-kie-kogito-serverless-operator/api/v1alpha08"
- "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles"
"github.com/apache/incubator-kie-kogito-serverless-operator/log"
)
-const (
- ConfigMapWorkflowPropsVolumeName = "workflow-properties"
- kogitoServiceUrlProperty = "kogito.service.url"
- kogitoServiceUrlProtocol = "http"
- dataIndexServiceUrlProperty = "mp.messaging.outgoing.kogito-processinstances-events.url"
- kafkaSmallRyeHealthProperty = "quarkus.smallrye-health.check.\"io.quarkus.kafka.client.health.KafkaHealthCheck\".enabled"
- dataIndexServiceUrlProtocol = "http"
-
- DataIndexImageBase = "quay.io/kiegroup/kogito-data-index-"
- DataIndexName = "data-index-service"
-
- PersistenceTypeEphemeral = "ephemeral"
- PersistenceTypePostgressql = "postgresql"
-
- microprofileServiceCatalogPropertyPrefix = "org.kie.kogito.addons.discovery."
- discoveryLikePropertyPattern = "^\\${(kubernetes|knative|openshift):(.*)}$"
+var (
+ immutableApplicationProperties = fmt.Sprintf("quarkus.http.port=%d\n"+
+ "quarkus.http.host=0.0.0.0\n"+
+ // We disable the Knative health checks to not block the dev pod to run if Knative objects are not available
+ // See: https://kiegroup.github.io/kogito-docs/serverlessworkflow/latest/eventing/consume-produce-events-with-knative-eventing.html#ref-knative-eventing-add-on-source-configuration
+ "org.kie.kogito.addons.knative.eventing.health-enabled=false\n"+
+ "quarkus.devservices.enabled=false\n"+
+ "quarkus.kogito.devservices.enabled=false\n", constants.DefaultHTTPWorkflowPortInt)
+
+ discoveryLikePropertyExpr = regexp.MustCompile(constants.DiscoveryLikePropertyPattern)
Review Comment:
This comment is related to the comment above regarding the DiscoveryLikePropertyPattern, note that it's really used here only.
##########
controllers/profiles/common/constants/platform_services.go:
##########
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package constants
+
+const (
+ ConfigMapWorkflowPropsVolumeName = "workflow-properties"
+ PersistenceTypePostgreSQL = "postgresql"
+ PersistenceTypeEphemeral = "ephemeral"
+
+ DataIndexServiceURLProperty = "mp.messaging.outgoing.kogito-processinstances-events.url"
+ DataIndexServiceURLProtocol = "http"
+
+ JobServiceURLProperty = "mp.messaging.outgoing.kogito-job-service-job-request-events.url"
+ JobServiceKafkaSinkInjectionHealthCheck = `quarkus.smallrye-health.check."org.kie.kogito.jobs.service.messaging.http.health.knative.KSinkInjectionHealthCheck".enabled`
+ JobServiceStatusChangeEventsProperty = "kogito.jobs-service.http.job-status-change-events"
+ JobServiceStatusChangeEventsURL = "mp.messaging.outgoing.kogito-job-service-job-status-events-http.url"
+ JobServiceURLProtocol = "http"
+ JobServiceDataSourceReactiveURLProperty = "quarkus.datasource.reactive.url"
+
+ KogitoServiceUrlProperty = "kogito.service.url"
+ KogitoServiceUrlProtocol = "http"
+ KafkaSmallRyeHealthProperty = "quarkus.smallrye-health.check.\"io.quarkus.kafka.client.health.KafkaHealthCheck\".enabled"
+
+ MicroprofileServiceCatalogPropertyPrefix = "org.kie.kogito.addons.discovery."
Review Comment:
This constant I'm not sure should belong to this file since is more specific to the workflow configurations and not related to the services.
##########
controllers/profiles/common/constants/platform_services.go:
##########
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package constants
+
+const (
+ ConfigMapWorkflowPropsVolumeName = "workflow-properties"
+ PersistenceTypePostgreSQL = "postgresql"
+ PersistenceTypeEphemeral = "ephemeral"
+
+ DataIndexServiceURLProperty = "mp.messaging.outgoing.kogito-processinstances-events.url"
+ DataIndexServiceURLProtocol = "http"
+
+ JobServiceURLProperty = "mp.messaging.outgoing.kogito-job-service-job-request-events.url"
+ JobServiceKafkaSinkInjectionHealthCheck = `quarkus.smallrye-health.check."org.kie.kogito.jobs.service.messaging.http.health.knative.KSinkInjectionHealthCheck".enabled`
+ JobServiceStatusChangeEventsProperty = "kogito.jobs-service.http.job-status-change-events"
+ JobServiceStatusChangeEventsURL = "mp.messaging.outgoing.kogito-job-service-job-status-events-http.url"
+ JobServiceURLProtocol = "http"
+ JobServiceDataSourceReactiveURLProperty = "quarkus.datasource.reactive.url"
+
+ KogitoServiceUrlProperty = "kogito.service.url"
+ KogitoServiceUrlProtocol = "http"
+ KafkaSmallRyeHealthProperty = "quarkus.smallrye-health.check.\"io.quarkus.kafka.client.health.KafkaHealthCheck\".enabled"
+
+ MicroprofileServiceCatalogPropertyPrefix = "org.kie.kogito.addons.discovery."
+ DiscoveryLikePropertyPattern = "^\\${(kubernetes|knative|openshift):(.*)}$"
Review Comment:
Same comment as above.
##########
controllers/platform/services/properties.go:
##########
@@ -0,0 +1,98 @@
+// Copyright 2023 Apache Software Foundation (ASF)
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package services
+
+import (
+ "fmt"
+ "strings"
+
+ operatorapi "github.com/apache/incubator-kie-kogito-serverless-operator/api/v1alpha08"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles/common/constants"
+
+ "github.com/magiconair/properties"
+)
+
+func generateReactiveURL(postgresSpec *operatorapi.PersistencePostgreSql, schema string, namespace string, dbName string, port int) string {
+ if len(postgresSpec.JdbcUrl) > 0 && strings.Contains(postgresSpec.JdbcUrl, "currentSchema=") {
+ return strings.Replace(strings.TrimPrefix(postgresSpec.JdbcUrl, "jdbc:"), "currentSchema=", "search_path=", -1)
+ }
+ databaseSchema := schema
+ if len(postgresSpec.ServiceRef.DatabaseSchema) > 0 {
+ databaseSchema = postgresSpec.ServiceRef.DatabaseSchema
+ }
+ databaseNamespace := namespace
+ if len(postgresSpec.ServiceRef.Namespace) > 0 {
+ databaseNamespace = postgresSpec.ServiceRef.Namespace
+ }
+ dataSourcePort := port
+ if postgresSpec.ServiceRef.Port != nil {
+ dataSourcePort = *postgresSpec.ServiceRef.Port
+ }
+ databaseName := dbName
+ if len(postgresSpec.ServiceRef.DatabaseName) > 0 {
+ databaseName = postgresSpec.ServiceRef.DatabaseName
+ }
+ return fmt.Sprintf("%s://%s:%d/%s?search_path=%s", constants.PersistenceTypePostgreSQL, postgresSpec.ServiceRef.Name+"."+databaseNamespace, dataSourcePort, databaseName, databaseSchema)
+}
+
+// GenerateDataIndexApplicationProperties returns the application properties required for the Data Index Service to work when deployed in a production profile
+// and data index' service's spec field `Enabled` set to true
+func GenerateDataIndexApplicationProperties(workflow *operatorapi.SonataFlow, platform *operatorapi.SonataFlowPlatform) *properties.Properties {
+ props := properties.NewProperties()
+ if profiles.IsProdProfile(workflow) && dataIndexEnabled(platform) {
+ di := NewDataIndexService(platform)
+ props.Set(
+ constants.DataIndexServiceURLProperty,
+ fmt.Sprintf("%s://%s.%s/processes", constants.DataIndexServiceURLProtocol, di.GetServiceName(), platform.Namespace),
+ )
+ }
+ return props
+}
+
+// GenerateJobServiceApplicationProperties returns the application properties required for the Job Service to work in a production profile and job service's
+// service's spec field `Enabled` set to true
+func GenerateJobServiceApplicationProperties(workflow *operatorapi.SonataFlow, platform *operatorapi.SonataFlowPlatform) *properties.Properties {
+ props := properties.NewProperties()
+ if profiles.IsProdProfile(workflow) && jobServiceEnabled(platform) {
Review Comment:
@jordigilh @ricardozanini In my opinion to instanciate the services even with no workflows is valid.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "ricardozanini (via GitHub)" <gi...@apache.org>.
ricardozanini commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1425490421
##########
controllers/profiles/common/properties/application.go:
##########
@@ -143,31 +131,18 @@ func (a *appPropertyHandler) Build() string {
func (a *appPropertyHandler) withKogitoServiceUrl() AppPropertyHandler {
var kogitoServiceUrl string
if len(a.workflow.Namespace) > 0 {
- kogitoServiceUrl = fmt.Sprintf("%s://%s.%s", kogitoServiceUrlProtocol, a.workflow.Name, a.workflow.Namespace)
+ kogitoServiceUrl = fmt.Sprintf("%s://%s.%s", constants.KogitoServiceUrlProtocol, a.workflow.Name, a.workflow.Namespace)
} else {
- kogitoServiceUrl = fmt.Sprintf("%s://%s", kogitoServiceUrlProtocol, a.workflow.Name)
+ kogitoServiceUrl = fmt.Sprintf("%s://%s", constants.KogitoServiceUrlProtocol, a.workflow.Name)
}
- return a.addDefaultMutableProperty(kogitoServiceUrlProperty, kogitoServiceUrl)
-}
-
-// withDataIndexServiceUrl adds the property dataIndexServiceUrlProperty to the application properties.
-// See Service Discovery https://kubernetes.io/docs/concepts/services-networking/service/#dns
-func (a *appPropertyHandler) withDataIndexServiceUrl() AppPropertyHandler {
- if profiles.IsProdProfile(a.workflow) && dataIndexEnabled(a.platform) {
Review Comment:
Only workflows not in `dev` profile in the given `SonataFlowPlatform` should be configured to communicate with Jobs Service and Data Index.
This is the use case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1425820348
##########
controllers/platform/services/properties.go:
##########
@@ -0,0 +1,131 @@
+// Copyright 2023 Apache Software Foundation (ASF)
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package services
+
+import (
+ "fmt"
+ "net/url"
+ "strings"
+
+ operatorapi "github.com/apache/incubator-kie-kogito-serverless-operator/api/v1alpha08"
+ "github.com/apache/incubator-kie-kogito-serverless-operator/controllers/profiles/common/constants"
+
+ "github.com/magiconair/properties"
+)
+
+func generateReactiveURL(postgresSpec *operatorapi.PersistencePostgreSql, schema string, namespace string, dbName string, port int) (string, error) {
+ if len(postgresSpec.JdbcUrl) > 0 {
+ s := strings.TrimLeft(postgresSpec.JdbcUrl, "jdbc:")
+ u, err := url.Parse(s)
+ if err != nil {
+ return "", err
+ }
+ ret := fmt.Sprintf("%s://", u.Scheme)
+ if len(u.User.Username()) > 0 {
+ p, ok := u.User.Password()
+ if ok {
+ ret = fmt.Sprintf("%s%s:%s@", ret, u.User.Username(), p)
+ }
+ }
+ ret = fmt.Sprintf("%s%s%s?", ret, u.Host, u.Path)
+ kv, err := url.ParseQuery(u.RawQuery)
+ if err != nil {
+ return "", err
+ }
+ spv := schema
+ if v, ok := kv["search_path"]; ok {
+ for _, val := range v {
+ if len(val) != 0 {
+ spv = v[0]
+ }
+ }
+ } else if v, ok := kv["currentSchema"]; ok {
+ for _, val := range v {
+ if len(val) != 0 {
+ spv = v[0]
+ }
+ }
+ }
+ return fmt.Sprintf("%ssearch_path=%s", ret, spv), nil
Review Comment:
If it can't infer the schema, I'll return the URL without the `search_path` parameter. Is that correct?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1425822947
##########
controllers/profiles/common/properties/application.go:
##########
@@ -143,31 +131,18 @@ func (a *appPropertyHandler) Build() string {
func (a *appPropertyHandler) withKogitoServiceUrl() AppPropertyHandler {
var kogitoServiceUrl string
if len(a.workflow.Namespace) > 0 {
- kogitoServiceUrl = fmt.Sprintf("%s://%s.%s", kogitoServiceUrlProtocol, a.workflow.Name, a.workflow.Namespace)
+ kogitoServiceUrl = fmt.Sprintf("%s://%s.%s", constants.KogitoServiceUrlProtocol, a.workflow.Name, a.workflow.Namespace)
} else {
- kogitoServiceUrl = fmt.Sprintf("%s://%s", kogitoServiceUrlProtocol, a.workflow.Name)
+ kogitoServiceUrl = fmt.Sprintf("%s://%s", constants.KogitoServiceUrlProtocol, a.workflow.Name)
}
- return a.addDefaultMutableProperty(kogitoServiceUrlProperty, kogitoServiceUrl)
-}
-
-// withDataIndexServiceUrl adds the property dataIndexServiceUrlProperty to the application properties.
-// See Service Discovery https://kubernetes.io/docs/concepts/services-networking/service/#dns
-func (a *appPropertyHandler) withDataIndexServiceUrl() AppPropertyHandler {
- if profiles.IsProdProfile(a.workflow) && dataIndexEnabled(a.platform) {
Review Comment:
As agreed, I'll revert the change and add back the conditional to check if the workflow is not nil and if the profile is not DEV.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "tchughesiv (via GitHub)" <gi...@apache.org>.
tchughesiv commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1408672750
##########
controllers/profiles/common/app_properties_test.go:
##########
@@ -128,7 +128,7 @@ func Test_appPropertyHandler_WithServicesWithUserOverrides(t *testing.T) {
generatedProps, propsErr = properties.LoadString(props)
assert.NoError(t, propsErr)
assert.Equal(t, 9, len(generatedProps.Keys()))
- assert.Equal(t, "http://"+platform.Name+"-"+DataIndexName+"."+platform.Namespace+"/processes", generatedProps.GetString(dataIndexServiceUrlProperty, ""))
+ assert.Equal(t, "http://"+platform.Name+"-"+DataIndexServiceName+"."+platform.Namespace+"/processes", generatedProps.GetString(dataIndexServiceUrlProperty, ""))
Review Comment:
may want tests that check each configured service separately and then together
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "ricardozanini (via GitHub)" <gi...@apache.org>.
ricardozanini commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1416034903
##########
controllers/platform/services.go:
##########
@@ -99,27 +98,9 @@ func createDataIndexDeployment(ctx context.Context, client client.Client, platfo
liveProbe := readyProbe.DeepCopy()
liveProbe.ProbeHandler.HTTPGet.Path = common.QuarkusHealthPathLive
dataDeployContainer := &corev1.Container{
Review Comment:
Are you talking about the env var? I'm sure if I followed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "tchughesiv (via GitHub)" <gi...@apache.org>.
tchughesiv commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1416175525
##########
controllers/platform/services.go:
##########
@@ -58,30 +55,32 @@ func (action *serviceAction) Handle(ctx context.Context, platform *operatorapi.S
return nil, err
}
- if err := createDataIndexComponents(ctx, action.client, platform); err != nil {
- return nil, err
+ if platform.Spec.Services.DataIndex != nil {
Review Comment:
@ricardozanini reviewing old notes i had on this, we discussed possibly scaling down a service to 0 when `enabled` was set to `false`. currently, however, the operator merely removes the `mp.messaging.outgoing.kogito-processinstances-events.url` from new / changed prod workflows.
i think that, during development, i realized scaling down to 0 would be dangerous considering that connections from existing workflows would break.
If the end-goal is indeed to scale the service down when `enabled: false`, we'd first need to revisit a separate enhancement request from @masayag around restarting affected workflow pods when platform configs are changed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1858292651
```
kogito.service.url (cluster dns named url to self e.g. http://jobs-service-postgresql/)
quarkus.datasource.jdbc.url
quarkus.datasource.reactive.url
health.check."org.kie.kogito.jobs.service.messaging.http.health.knative.KSinkInjectionHealthCheck".enabled=false
```
@wmedvede I noticed that the value for this property is already added as part of the environment values when defining the persistence. Should it be duplicated in the `application.properties`?
So far, only 2 properties seem to be generated from the job service itself, and that's the reactive URL and the health check. The kogito service URL is already added and the JDBC URL is added in the environment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "wmedvede (via GitHub)" <gi...@apache.org>.
wmedvede commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1855616654
**Case2:**
Given the platform of Case1, in a later point in time I try to deploy a workflow.
Reproducer here:
These are the generated properties for the workflow:
**here we can see:**
quarkus.datasource.reactive.url = postgresql://postgres:5432/sonataflow?search_path=jobs-service (is not for the workflow, but for the jobs-service)
quarkus.smallrye-health.check.\"org.kie.kogito.jobs.service.messaging.http.health.knative.KSinkInjectionHealthCheck\".enabled = false (is not for the workflow, but for the jobs-service)
**Some missing properties for the workflow: (These ones are, this was my fault, I forgot to mention this ones :( )**
mp.messaging.outgoing.kogito-job-service-job-request-events.connector=quarkus-http
kogito.events.processdefinitions.enabled=false
kogito.events.usertasks.enabled=false
kogito.events.variables.enabled=false
**Remainder:**
If we detect that the Data Index is present, then we must also generate this properties for the workflow. I haven't tested this last,
mp.messaging.outgoing.kogito-processinstances-events.url=http://<data index service correct url>/processes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "tchughesiv (via GitHub)" <gi...@apache.org>.
tchughesiv commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1856488517
> It's fixed by increasing the `maxProperty` value to 3. @tchughesiv do you know why it was limited to 2 max in the first place? Any reason for the validation? Can it be increased to 3?
No, it should not be increased to 3. The validation is set this way because `ServiceRef` and `JdbcUrl` should not be set simultaneously. It's an either/or scenario. `SecretRef` is a required field.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1856521935
> > It's fixed by increasing the `maxProperty` value to 3. @tchughesiv do you know why it was limited to 2 max in the first place? Any reason for the validation? Can it be increased to 3?
>
> No, it should not be increased to 3. The validation is set this way because `ServiceRef` and `JdbcUrl` should not be set simultaneously. It's an either/or scenario. `SecretRef` is a required field.
>
> @jordigilh can you show me the platform CR you're trying to use?
The problem is that the `ServiceRef` field is not a pointer and it's a structure, so it is populated with default values and that's good enough for the schema validation to be considered as populated. That's why, when providing the JDBC URL, it considers all 3 fields as populated and fails the validation.
By changing the `ServiceRef` to be a pointer, and adjusting the code to take that into account, solves the issue without having to change the limits:
```
// Service reference to postgresql datasource. Mutually exclusive to jdbcUrl.
// +optional
ServiceRef *PostgreSqlServiceOptions `json:"serviceRef,omitempty"`
```
You can reproduce the error by creating this test case:
https://github.com/wmedvede/techpreview2/tree/test-platform-services/platforms/jobservice_as_platform_service_postgresql
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "jordigilh (via GitHub)" <gi...@apache.org>.
jordigilh commented on code in PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#discussion_r1418285088
##########
controllers/profiles/common/app_properties.go:
##########
@@ -38,34 +43,313 @@ import (
)
const (
- ConfigMapWorkflowPropsVolumeName = "workflow-properties"
- kogitoServiceUrlProperty = "kogito.service.url"
- kogitoServiceUrlProtocol = "http"
- dataIndexServiceUrlProperty = "mp.messaging.outgoing.kogito-processinstances-events.url"
- kafkaSmallRyeHealthProperty = "quarkus.smallrye-health.check.\"io.quarkus.kafka.client.health.KafkaHealthCheck\".enabled"
- dataIndexServiceUrlProtocol = "http"
-
- DataIndexImageBase = "quay.io/kiegroup/kogito-data-index-"
- DataIndexName = "data-index-service"
-
- PersistenceTypeEphemeral = "ephemeral"
- PersistenceTypePostgressql = "postgresql"
-
+ ConfigMapWorkflowPropsVolumeName = "workflow-properties"
+ kogitoServiceUrlProperty = "kogito.service.url"
+ kogitoServiceUrlProtocol = "http"
+ dataIndexServiceUrlProperty = "mp.messaging.outgoing.kogito-processinstances-events.url"
+ kafkaSmallRyeHealthProperty = "quarkus.smallrye-health.check.\"io.quarkus.kafka.client.health.KafkaHealthCheck\".enabled"
+ dataIndexServiceUrlProtocol = "http"
+ jobServiceURLProperty = "mp.messaging.outgoing.kogito-job-service-job-request-events.url"
+ jobServiceKafkaSinkInjectionHealthCheck = `quarkus.smallrye-health.check."org.kie.kogito.jobs.service.messaging.http.health.knative.KSinkInjectionHealthCheck".enabled`
+ jobServiceStatusChangeEventsProperty = "kogito.jobs-service.http.job-status-change-events"
+ jobServiceStatusChangeEventsURL = "mp.messaging.outgoing.kogito-job-service-job-status-events-http.url"
+ jobServiceURLProtocol = "http"
+ jobServiceDataSourceReactiveURLProperty = "quarkus.datasource.reactive.url"
+
+ imageNamePrefix = "quay.io/kiegroup/kogito"
+ DataIndexServiceName = "data-index-service"
+ DataIndexName = "data-index"
+ JobServiceName = "jobs-service"
+ PersistenceTypeEphemeral = "ephemeral"
+ PersistenceTypePostgressql = "postgresql"
microprofileServiceCatalogPropertyPrefix = "org.kie.kogito.addons.discovery."
discoveryLikePropertyPattern = "^\\${(kubernetes|knative|openshift):(.*)}$"
+
+ defaultDatabaseName string = "sonataflow"
+ defaultPostgreSQLPort int = 5432
+)
Review Comment:
Done. I'm not sure if I got it right though. There are a lot of inter dependencies in the handler and the platform services that caused a import cycle problem when I tried to keep things together. Eventually I found a way to make a clear cut by defining a package inside `platform` and `profile` that allowed me to call the platform service properties inside the profile handler like it's being done now.
If what you expect is to duplicate the handler struct and functions for `platform` and `profiles`, with the caveat that only in `platform` it will inject the platform services properties, let me know.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "ricardozanini (via GitHub)" <gi...@apache.org>.
ricardozanini commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1862624150
@wmedvede look at @jordigilh's comments. The image doesn't have the runtimes fabric8 discovery anymore, so the properties in this test case must be updated.
@jordigilh I'll fix it in a separate PR, you can ignore it for the moment. After a meeting I'm having now, I'll fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [KOGITO-9940] Add Job Service to SonataFlowPlatform CRD [incubator-kie-kogito-serverless-operator]
Posted by "ricardozanini (via GitHub)" <gi...@apache.org>.
ricardozanini commented on PR #311:
URL: https://github.com/apache/incubator-kie-kogito-serverless-operator/pull/311#issuecomment-1860585480
@tchughesiv @wmedvede
> If the jobs service is NOT installed, we still set the url, and we invent it :), to http://jobs-service/v2/jobs/events, np, we can manage in another PR to fine tune it. And, if the jobs service is installed we set the correct cluster dns url)
@tchughesiv I thought you meant the property on the workflow side to connect to JS. Yes, in this case, a property set in JS should be pointed to localhost instead within the same pod. `localhost` should do.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org