You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by pc...@apache.org on 2024/01/09 14:21:54 UTC
(camel-k) 06/08: doc: synthetic Integrations
This is an automated email from the ASF dual-hosted git repository.
pcongiusti pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel-k.git
commit a1601b7e04b3a5ec36a180de522546bb5bd7ec81
Author: Pasquale Congiusti <pa...@gmail.com>
AuthorDate: Tue Dec 5 10:15:21 2023 +0100
doc: synthetic Integrations
---
config/manager/operator-deployment.yaml | 3 +
docs/modules/ROOT/nav.adoc | 3 +-
docs/modules/ROOT/pages/running/import.adoc | 44 +++++++++
e2e/common/synthetic/default.go | 26 ------
e2e/common/synthetic/synthetic_test.go | 93 -------------------
.../files/deploy.yaml | 0
e2e/commonwithcustominstall/synthetic_test.go | 100 +++++++++++++++++++++
pkg/cmd/operator/operator.go | 9 +-
script/Makefile | 1 -
9 files changed, 156 insertions(+), 123 deletions(-)
diff --git a/config/manager/operator-deployment.yaml b/config/manager/operator-deployment.yaml
index ae47c6e29..f44e150d5 100644
--- a/config/manager/operator-deployment.yaml
+++ b/config/manager/operator-deployment.yaml
@@ -72,6 +72,9 @@ spec:
valueFrom:
fieldRef:
fieldPath: metadata.namespace
+ # Change to true to be able to create synthetic Integrations
+ - name: CAMEL_K_SYNTHETIC_INTEGRATIONS
+ value: "false"
livenessProbe:
httpGet:
path: /healthz
diff --git a/docs/modules/ROOT/nav.adoc b/docs/modules/ROOT/nav.adoc
index 017b36dfc..20fb04f30 100644
--- a/docs/modules/ROOT/nav.adoc
+++ b/docs/modules/ROOT/nav.adoc
@@ -20,8 +20,9 @@
** xref:running/dev-mode.adoc[Developer mode]
** xref:running/dry-run.adoc[Dry run]
** xref:running/runtime-version.adoc[Camel version]
-** xref:running/camel-runtimes.adoc[Camel runtimes]
** xref:running/quarkus-native.adoc[Quarkus Native]
+** xref:running/camel-runtimes.adoc[Camel runtimes]
+** xref:running/import.adoc[Import existing Camel apps]
** xref:running/run-from-github.adoc[Run from GitHub]
** xref:running/promoting.adoc[Promote an Integration]
** xref:running/knative-sink.adoc[Knative Sinks]
diff --git a/docs/modules/ROOT/pages/running/import.adoc b/docs/modules/ROOT/pages/running/import.adoc
new file mode 100644
index 000000000..c09552121
--- /dev/null
+++ b/docs/modules/ROOT/pages/running/import.adoc
@@ -0,0 +1,44 @@
+= Importing existing Camel applications
+
+You may have already a Camel application running on your cluster. You may have created it via a manual deployment, a CICD or any other deployment mechanism you have in place. Since the Camel K operator is meant to operate any Camel application out there, then, you will be able to import it and monitor in a similar fashion of any other Camel K **managed Integration**.
+
+This feature is disabled by default. In order to enable it, you need to run the operator deployment with an environment variable, `CAMEL_K_SYNTHETIC_INTEGRATIONS`, set to `true`.
+
+NOTE: you will be only able to monitor the synthetic Integrations. Camel K won't be able to alter the lifecycle of non managed Integrations (ie, rebuild the original application).
+
+It's important to notice that the operator won't be altering any field of the original application in order to avoid breaking any deployment procedure which is already in place. As it cannot make any assumption on the way the application is built and deployed, it will only be able to **watch** for any changes happening around it.
+
+[[deploy-and-monitor]]
+== Deploy externally, monitor via Camel K Operator
+
+An imported Integration is known as **synthetic Integration**. You can import any Camel application deployed as a **Deployment**, **CronJob** or **Knative Service**. We control this behavior via a label (`camel.apache.org/integration`) that the user need to apply on the Camel application (either manually or introducing in the deployment process, ie, via CICD).
+
+NOTE: the example here will work in a similar way using CronJob and Knative Service.
+
+As an example, we show how to import a Camel application which was deployed with the Deployment kind. Let's assume it is called `my-deploy`.
+```
+$ kubectl label deploy my-camel-sb-svc camel.apache.org/integration=my-it
+```
+The operator immediately creates a synthetic Integration:
+```
+$ kubectl get it
+NAMESPACE NAME PHASE RUNTIME PROVIDER RUNTIME VERSION KIT REPLICAS
+test-79c385c3-d58e-4c28-826d-b14b6245f908 my-it Cannot Monitor Pods
+```
+You can see it will be in `Cannot Monitor Pods` status phase. This is expected because the way Camel K operator monitor Pods. It requires that the same label applied to the Deployment is inherited by the generated Pods. For this reason, beside labelling the Deployment, we need to add a label in the Deployment template.
+```
+$ kubectl patch deployment my-camel-sb-svc --patch '{"spec": {"template": {"metadata": {"labels": {"camel.apache.org/integration": "my-it"}}}}}'
+```
+Also this operator can be performed manually or automated in the deployment procedure. We can see now that the operator will be able to monitor accordingly the status of the Pods:
+```
+$ kubectl get it
+NAMESPACE NAME PHASE RUNTIME PROVIDER RUNTIME VERSION KIT REPLICAS
+test-79c385c3-d58e-4c28-826d-b14b6245f908 my-it Running 1
+```
+From now on, you will be able to monitor the status of the synthetic Integration in a similar fashion of what you do with managed Integrations. If, for example, your Deployment will scale up or down, then, you will see this information reflecting accordingly:
+```
+$ kubectl scale deployment my-camel-sb-svc --replicas 2
+$ kubectl get it
+NAMESPACE NAME PHASE RUNTIME PROVIDER RUNTIME VERSION KIT REPLICAS
+test-79c385c3-d58e-4c28-826d-b14b6245f908 my-it Running 2
+```
diff --git a/e2e/common/synthetic/default.go b/e2e/common/synthetic/default.go
deleted file mode 100644
index a7e504e46..000000000
--- a/e2e/common/synthetic/default.go
+++ /dev/null
@@ -1,26 +0,0 @@
-//go:build integration
-// +build integration
-
-/*
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements. See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package synthetic
-
-import "github.com/apache/camel-k/v2/e2e/support"
-
-var ns = support.GetEnvOrDefault("CAMEL_K_TEST_NAMESPACE", support.GetCIProcessID())
-var operatorID = support.GetEnvOrDefault("CAMEL_K_OPERATOR_ID", support.GetCIProcessID())
diff --git a/e2e/common/synthetic/synthetic_test.go b/e2e/common/synthetic/synthetic_test.go
deleted file mode 100644
index f9f1532ce..000000000
--- a/e2e/common/synthetic/synthetic_test.go
+++ /dev/null
@@ -1,93 +0,0 @@
-//go:build integration
-// +build integration
-
-// To enable compilation of this file in Goland, go to "Settings -> Go -> Vendoring & Build Tags -> Custom Tags" and add "integration"
-
-/*
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements. See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package synthetic
-
-import (
- "testing"
-
- . "github.com/onsi/gomega"
-
- . "github.com/apache/camel-k/v2/e2e/support"
- v1 "github.com/apache/camel-k/v2/pkg/apis/camel/v1"
- . "github.com/onsi/gomega/gstruct"
- appsv1 "k8s.io/api/apps/v1"
- corev1 "k8s.io/api/core/v1"
-)
-
-func TestSyntheticIntegrationFromDeployment(t *testing.T) {
- RegisterTestingT(t)
-
- // Run the external deployment
- ExpectExecSucceed(t, Kubectl("apply", "-f", "files/deploy.yaml", "-n", ns))
- Eventually(DeploymentCondition(ns, "my-camel-sb-svc", appsv1.DeploymentProgressing), TestTimeoutShort).
- Should(MatchFields(IgnoreExtras, Fields{
- "Status": Equal(corev1.ConditionTrue),
- "Reason": Equal("NewReplicaSetAvailable"),
- }))
-
- // Label the deployment --> Verify the Integration is created (cannot still monitor)
- ExpectExecSucceed(t, Kubectl("label", "deploy", "my-camel-sb-svc", "camel.apache.org/integration=my-it", "-n", ns))
- Eventually(IntegrationPhase(ns, "my-it"), TestTimeoutShort).Should(Equal(v1.IntegrationPhaseCannotMonitor))
- Eventually(IntegrationConditionStatus(ns, "my-it", v1.IntegrationConditionReady), TestTimeoutShort).Should(Equal(corev1.ConditionTrue))
-
- // Label the deployment template --> Verify the Integration is monitored
- ExpectExecSucceed(t, Kubectl("patch", "deployment", "my-camel-sb-svc", "--patch", `{"spec": {"template": {"metadata": {"labels": {"camel.apache.org/integration": "my-it"}}}}}`, "-n", ns))
- Eventually(IntegrationPhase(ns, "my-it"), TestTimeoutShort).Should(Equal(v1.IntegrationPhaseRunning))
- Eventually(IntegrationConditionStatus(ns, "my-it", v1.IntegrationConditionReady), TestTimeoutShort).Should(Equal(corev1.ConditionTrue))
- one := int32(1)
- Eventually(IntegrationStatusReplicas(ns, "my-it"), TestTimeoutShort).Should(Equal(&one))
-
- // Delete the deployment --> Verify the Integration is in missing status
- ExpectExecSucceed(t, Kubectl("delete", "deploy", "my-camel-sb-svc", "-n", ns))
- Eventually(IntegrationPhase(ns, "my-it"), TestTimeoutShort).Should(Equal(v1.IntegrationPhaseImportMissing))
- Eventually(IntegrationConditionStatus(ns, "my-it", v1.IntegrationConditionReady), TestTimeoutShort).Should(Equal(corev1.ConditionFalse))
- zero := int32(0)
- Eventually(IntegrationStatusReplicas(ns, "my-it"), TestTimeoutShort).Should(Equal(&zero))
-
- // Recreate the deployment and label --> Verify the Integration is monitored
- ExpectExecSucceed(t, Kubectl("apply", "-f", "files/deploy.yaml", "-n", ns))
- ExpectExecSucceed(t, Kubectl("label", "deploy", "my-camel-sb-svc", "camel.apache.org/integration=my-it", "-n", ns))
- ExpectExecSucceed(t, Kubectl("patch", "deployment", "my-camel-sb-svc", "--patch", `{"spec": {"template": {"metadata": {"labels": {"camel.apache.org/integration": "my-it"}}}}}`, "-n", ns))
- Eventually(IntegrationPhase(ns, "my-it"), TestTimeoutShort).Should(Equal(v1.IntegrationPhaseRunning))
- Eventually(IntegrationConditionStatus(ns, "my-it", v1.IntegrationConditionReady), TestTimeoutShort).Should(Equal(corev1.ConditionTrue))
- Eventually(IntegrationStatusReplicas(ns, "my-it"), TestTimeoutShort).Should(Equal(&one))
-
- // Remove label from the deployment --> Verify the Integration is deleted
- ExpectExecSucceed(t, Kubectl("label", "deploy", "my-camel-sb-svc", "camel.apache.org/integration-", "-n", ns))
- Eventually(Integration(ns, "my-it"), TestTimeoutShort).Should(BeNil())
-
- // Add label back to the deployment --> Verify the Integration is created
- ExpectExecSucceed(t, Kubectl("label", "deploy", "my-camel-sb-svc", "camel.apache.org/integration=my-it", "-n", ns))
- Eventually(IntegrationPhase(ns, "my-it"), TestTimeoutShort).Should(Equal(v1.IntegrationPhaseRunning))
- Eventually(IntegrationConditionStatus(ns, "my-it", v1.IntegrationConditionReady), TestTimeoutShort).Should(Equal(corev1.ConditionTrue))
- Eventually(IntegrationStatusReplicas(ns, "my-it"), TestTimeoutShort).Should(Equal(&one))
- // Scale the deployment --> verify replicas are correctly set
- ExpectExecSucceed(t, Kubectl("scale", "deploy", "my-camel-sb-svc", "--replicas", "2", "-n", ns))
- two := int32(2)
- Eventually(IntegrationStatusReplicas(ns, "my-it"), TestTimeoutShort).Should(Equal(&two))
-
- // Delete Integration and deployments --> verify no Integration exists any longer
- Expect(Kamel("delete", "--all", "-n", ns).Execute()).To(Succeed())
- ExpectExecSucceed(t, Kubectl("delete", "deploy", "my-camel-sb-svc", "-n", ns))
- Eventually(Integration(ns, "my-it"), TestTimeoutShort).Should(BeNil())
-}
diff --git a/e2e/common/synthetic/files/deploy.yaml b/e2e/commonwithcustominstall/files/deploy.yaml
similarity index 100%
rename from e2e/common/synthetic/files/deploy.yaml
rename to e2e/commonwithcustominstall/files/deploy.yaml
diff --git a/e2e/commonwithcustominstall/synthetic_test.go b/e2e/commonwithcustominstall/synthetic_test.go
new file mode 100644
index 000000000..a1b92f40e
--- /dev/null
+++ b/e2e/commonwithcustominstall/synthetic_test.go
@@ -0,0 +1,100 @@
+//go:build integration
+// +build integration
+
+// To enable compilation of this file in Goland, go to "Settings -> Go -> Vendoring & Build Tags -> Custom Tags" and add "integration"
+
+/*
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package commonwithcustominstall
+
+import (
+ "testing"
+
+ . "github.com/onsi/gomega"
+
+ . "github.com/apache/camel-k/v2/e2e/support"
+ v1 "github.com/apache/camel-k/v2/pkg/apis/camel/v1"
+ . "github.com/onsi/gomega/gstruct"
+ appsv1 "k8s.io/api/apps/v1"
+ corev1 "k8s.io/api/core/v1"
+)
+
+func TestSyntheticIntegrationFromDeployment(t *testing.T) {
+ RegisterTestingT(t)
+ WithNewTestNamespace(t, func(ns string) {
+ // Install Camel K with the synthetic Integration feature variable
+ operatorID := "camel-k-synthetic-env"
+ Expect(KamelInstallWithID(operatorID, ns,
+ "--operator-env-vars", "CAMEL_K_SYNTHETIC_INTEGRATIONS=true",
+ ).Execute()).To(Succeed())
+
+ // Run the external deployment
+ ExpectExecSucceed(t, Kubectl("apply", "-f", "files/deploy.yaml", "-n", ns))
+ Eventually(DeploymentCondition(ns, "my-camel-sb-svc", appsv1.DeploymentProgressing), TestTimeoutShort).
+ Should(MatchFields(IgnoreExtras, Fields{
+ "Status": Equal(corev1.ConditionTrue),
+ "Reason": Equal("NewReplicaSetAvailable"),
+ }))
+
+ // Label the deployment --> Verify the Integration is created (cannot still monitor)
+ ExpectExecSucceed(t, Kubectl("label", "deploy", "my-camel-sb-svc", "camel.apache.org/integration=my-it", "-n", ns))
+ Eventually(IntegrationPhase(ns, "my-it"), TestTimeoutShort).Should(Equal(v1.IntegrationPhaseCannotMonitor))
+ Eventually(IntegrationConditionStatus(ns, "my-it", v1.IntegrationConditionReady), TestTimeoutShort).Should(Equal(corev1.ConditionTrue))
+
+ // Label the deployment template --> Verify the Integration is monitored
+ ExpectExecSucceed(t, Kubectl("patch", "deployment", "my-camel-sb-svc", "--patch", `{"spec": {"template": {"metadata": {"labels": {"camel.apache.org/integration": "my-it"}}}}}`, "-n", ns))
+ Eventually(IntegrationPhase(ns, "my-it"), TestTimeoutShort).Should(Equal(v1.IntegrationPhaseRunning))
+ Eventually(IntegrationConditionStatus(ns, "my-it", v1.IntegrationConditionReady), TestTimeoutShort).Should(Equal(corev1.ConditionTrue))
+ one := int32(1)
+ Eventually(IntegrationStatusReplicas(ns, "my-it"), TestTimeoutShort).Should(Equal(&one))
+
+ // Delete the deployment --> Verify the Integration is in missing status
+ ExpectExecSucceed(t, Kubectl("delete", "deploy", "my-camel-sb-svc", "-n", ns))
+ Eventually(IntegrationPhase(ns, "my-it"), TestTimeoutShort).Should(Equal(v1.IntegrationPhaseImportMissing))
+ Eventually(IntegrationConditionStatus(ns, "my-it", v1.IntegrationConditionReady), TestTimeoutShort).Should(Equal(corev1.ConditionFalse))
+ zero := int32(0)
+ Eventually(IntegrationStatusReplicas(ns, "my-it"), TestTimeoutShort).Should(Equal(&zero))
+
+ // Recreate the deployment and label --> Verify the Integration is monitored
+ ExpectExecSucceed(t, Kubectl("apply", "-f", "files/deploy.yaml", "-n", ns))
+ ExpectExecSucceed(t, Kubectl("label", "deploy", "my-camel-sb-svc", "camel.apache.org/integration=my-it", "-n", ns))
+ ExpectExecSucceed(t, Kubectl("patch", "deployment", "my-camel-sb-svc", "--patch", `{"spec": {"template": {"metadata": {"labels": {"camel.apache.org/integration": "my-it"}}}}}`, "-n", ns))
+ Eventually(IntegrationPhase(ns, "my-it"), TestTimeoutShort).Should(Equal(v1.IntegrationPhaseRunning))
+ Eventually(IntegrationConditionStatus(ns, "my-it", v1.IntegrationConditionReady), TestTimeoutShort).Should(Equal(corev1.ConditionTrue))
+ Eventually(IntegrationStatusReplicas(ns, "my-it"), TestTimeoutShort).Should(Equal(&one))
+
+ // Remove label from the deployment --> Verify the Integration is deleted
+ ExpectExecSucceed(t, Kubectl("label", "deploy", "my-camel-sb-svc", "camel.apache.org/integration-", "-n", ns))
+ Eventually(Integration(ns, "my-it"), TestTimeoutShort).Should(BeNil())
+
+ // Add label back to the deployment --> Verify the Integration is created
+ ExpectExecSucceed(t, Kubectl("label", "deploy", "my-camel-sb-svc", "camel.apache.org/integration=my-it", "-n", ns))
+ Eventually(IntegrationPhase(ns, "my-it"), TestTimeoutShort).Should(Equal(v1.IntegrationPhaseRunning))
+ Eventually(IntegrationConditionStatus(ns, "my-it", v1.IntegrationConditionReady), TestTimeoutShort).Should(Equal(corev1.ConditionTrue))
+ Eventually(IntegrationStatusReplicas(ns, "my-it"), TestTimeoutShort).Should(Equal(&one))
+ // Scale the deployment --> verify replicas are correctly set
+ ExpectExecSucceed(t, Kubectl("scale", "deploy", "my-camel-sb-svc", "--replicas", "2", "-n", ns))
+ two := int32(2)
+ Eventually(IntegrationStatusReplicas(ns, "my-it"), TestTimeoutShort).Should(Equal(&two))
+
+ // Delete Integration and deployments --> verify no Integration exists any longer
+ Expect(Kamel("delete", "--all", "-n", ns).Execute()).To(Succeed())
+ ExpectExecSucceed(t, Kubectl("delete", "deploy", "my-camel-sb-svc", "-n", ns))
+ Eventually(Integration(ns, "my-it"), TestTimeoutShort).Should(BeNil())
+ })
+}
diff --git a/pkg/cmd/operator/operator.go b/pkg/cmd/operator/operator.go
index 12edd7cc1..97996f48c 100644
--- a/pkg/cmd/operator/operator.go
+++ b/pkg/cmd/operator/operator.go
@@ -232,8 +232,13 @@ func Run(healthPort, monitoringPort int32, leaderElection bool, leaderElectionID
install.OperatorStartupOptionalTools(installCtx, bootstrapClient, watchNamespace, operatorNamespace, log)
exitOnError(findOrCreateIntegrationPlatform(installCtx, bootstrapClient, operatorNamespace), "failed to create integration platform")
- log.Info("Starting the synthetic Integration manager")
- exitOnError(synthetic.ManageSyntheticIntegrations(ctx, ctrlClient, mgr.GetCache(), mgr.GetAPIReader()), "synthetic Integration manager error")
+ synthEnvVal, synth := os.LookupEnv("CAMEL_K_SYNTHETIC_INTEGRATIONS")
+ if synth && synthEnvVal == "true" {
+ log.Info("Starting the synthetic Integration manager")
+ exitOnError(synthetic.ManageSyntheticIntegrations(ctx, ctrlClient, mgr.GetCache(), mgr.GetAPIReader()), "synthetic Integration manager error")
+ } else {
+ log.Info("Synthetic Integration manager not configured, skipping")
+ }
log.Info("Starting the manager")
exitOnError(mgr.Start(ctx), "manager exited non-zero")
}
diff --git a/script/Makefile b/script/Makefile
index 282357f8b..be5391bd4 100644
--- a/script/Makefile
+++ b/script/Makefile
@@ -262,7 +262,6 @@ test-common: do-build
go test -timeout 30m -v ./e2e/common/misc -tags=integration $(TEST_INTEGRATION_COMMON_LANG_RUN) $(GOTESTFMT) || FAILED=1; \
go test -timeout 60m -v ./e2e/common/traits -tags=integration $(TEST_INTEGRATION_COMMON_LANG_RUN) $(GOTESTFMT) || FAILED=1; \
go test -timeout 20m -v ./e2e/common/runtimes -tags=integration $(TEST_INTEGRATION_COMMON_LANG_RUN) $(GOTESTFMT) || FAILED=1; \
- go test -timeout 10m -v ./e2e/common/synthetic -tags=integration $(TEST_INTEGRATION_COMMON_LANG_RUN) $(GOTESTFMT) || FAILED=1; \
go test -timeout 10m -v ./e2e/common/support/teardown_test.go -tags=integration $(TEST_INTEGRATION_COMMON_LANG_RUN) $(GOTESTFMT) || FAILED=1; \
exit $${FAILED}