You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by gy...@apache.org on 2022/05/22 05:43:47 UTC
[flink-kubernetes-operator] branch main updated (c54c7bf -> df8a1ad)
This is an automated email from the ASF dual-hosted git repository.
gyfora pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/flink-kubernetes-operator.git
from c54c7bf [hotfix] Allow last-state restore of terminal cluster + improve logging
new 233fe97 [FLINK-27717][FLINK-27302] Document upgrade process and compatibility guarantees
new df8a1ad [FLINK-27644] Upgrade resource overview page
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
README.md | 17 ++++---
docs/content/docs/custom-resource/overview.md | 14 ++----
docs/content/docs/operations/compatibility.md | 63 +++++++++++++++++++++++++
docs/content/docs/operations/upgrade.md | 66 +++++++++++++++++++++++++--
4 files changed, 141 insertions(+), 19 deletions(-)
create mode 100644 docs/content/docs/operations/compatibility.md
[flink-kubernetes-operator] 02/02: [FLINK-27644] Upgrade resource overview page
Posted by gy...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
gyfora pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/flink-kubernetes-operator.git
commit df8a1ad71a18d5cf5c459b9ee27874c7cab6c533
Author: Gyula Fora <g_...@apple.com>
AuthorDate: Fri May 20 12:56:40 2022 +0200
[FLINK-27644] Upgrade resource overview page
---
README.md | 2 +-
docs/content/docs/custom-resource/overview.md | 14 +++++---------
2 files changed, 6 insertions(+), 10 deletions(-)
diff --git a/README.md b/README.md
index bc123db..ea7dc00 100644
--- a/README.md
+++ b/README.md
@@ -30,7 +30,7 @@ Please check out our docs to read about the [upgrade process](https://nightlies.
## Version Matrix
-| Operator Version | API Version | Flink Compatibility | Operator Image Tag |
+| Operator Version | API Version | Flink Compatibility | Operator Image |
|----------------------|-------------|---------------------|------------------------------------------|
| `latest` (main HEAD) | `v1beta1` | 1.13, 1.14, 1.15 | |
| `release-1.0.0` | `v1beta1` | 1.13, 1.14, 1.15 | Release process underway |
diff --git a/docs/content/docs/custom-resource/overview.md b/docs/content/docs/custom-resource/overview.md
index 175e5f0..f4ea786 100644
--- a/docs/content/docs/custom-resource/overview.md
+++ b/docs/content/docs/custom-resource/overview.md
@@ -67,20 +67,16 @@ metadata:
spec:
...
status:
+ clusterInfo:
+ ...
jobManagerDeploymentStatus: READY
jobStatus:
- jobId: 93dfe8199a35d5503f4048a1a999c704
- jobName: State machine job
- savepointInfo: {}
- state: RUNNING
- updateTime: "1647351134601"
+ ...
reconciliationStatus:
- lastReconciledSpec:
- ...
- success: true
+ ...
```
-Users can use the status of the FlinkDeployment to gauge the health of their deployments and any executed operation.
+While the status contains a lot of information that the operator tracks about the deployment, it is considered to be internal to the operator logic and users should only rely on it with care.
## FlinkDeployment spec overview
[flink-kubernetes-operator] 01/02: [FLINK-27717][FLINK-27302] Document upgrade process and compatibility guarantees
Posted by gy...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
gyfora pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/flink-kubernetes-operator.git
commit 233fe97a8faa0249772139bf65388a5ae5d19a4d
Author: Gyula Fora <g_...@apple.com>
AuthorDate: Fri May 20 12:12:35 2022 +0200
[FLINK-27717][FLINK-27302] Document upgrade process and compatibility guarantees
---
README.md | 17 ++++---
docs/content/docs/operations/compatibility.md | 63 +++++++++++++++++++++++++
docs/content/docs/operations/upgrade.md | 66 +++++++++++++++++++++++++--
3 files changed, 136 insertions(+), 10 deletions(-)
diff --git a/README.md b/README.md
index 5d40ab1..bc123db 100644
--- a/README.md
+++ b/README.md
@@ -13,25 +13,28 @@ Check our [quick-start](https://nightlies.apache.org/flink/flink-kubernetes-oper
## Features at a glance
- - Deploy and monitor Flink Application and Session deployments
+ - Deploy and monitor Flink Application, Session and Job deployments
- Upgrade, suspend and delete deployments
- Full logging and metrics integration
- Flexible deployments and native integration with Kubernetes tooling
+For the complete feature-set please refer to our [documentation](https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/docs/concepts/overview/).
+
## Project Status
-### Project status: beta
+### Project status: Production Ready
### Current API version: `v1beta1`
-If you are currently using the v1alpha1 version of the APIs in your manifests, please update them to use the v1beta1 version by changing apiVersion: "flink.apache.org/v1alpha1" to apiVersion: "flink.apache.org/v1beta1".
+Please check out our docs to read about the [upgrade process](https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/docs/operations/upgrade/) and our [backward compatibility guarantees](https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/docs/operations/compatibility/).
## Version Matrix
-| Operator Version | API Version | Flink Release | Operator Image Tag | Helm Chart Version |
-| ------------- | ------------- | ------------- | ------------- | ------------- |
-| `latest` (main HEAD) | `v1beta1` | 1.15 | | |
-| `release-0.1.0` | `v1alpha1` | 1.14 | `2c166e3` | 0.1.0 |
+| Operator Version | API Version | Flink Compatibility | Operator Image Tag |
+|----------------------|-------------|---------------------|------------------------------------------|
+| `latest` (main HEAD) | `v1beta1` | 1.13, 1.14, 1.15 | |
+| `release-1.0.0` | `v1beta1` | 1.13, 1.14, 1.15 | Release process underway |
+| `release-0.1.0` | `v1alpha1` | 1.14 | `apache/flink-kubernetes-operator:0.1.0` |
## Support
diff --git a/docs/content/docs/operations/compatibility.md b/docs/content/docs/operations/compatibility.md
new file mode 100644
index 0000000..b0922d6
--- /dev/null
+++ b/docs/content/docs/operations/compatibility.md
@@ -0,0 +1,63 @@
+---
+title: "Compatibility Guarantees"
+weight: 4
+type: docs
+aliases:
+- /operations/compatibility.html
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Compatibility Guarantees
+
+It is very important for users to clearly understand the compatibility guarantees the operator provides when upgrading to new versions.
+
+## Custom Resource backward compatibility
+
+The main user facing api of the operator are the Flink [custom resources]({{< ref "docs/custom-resource/overview" >}}).
+
+Starting from `v1beta1` (operator version 1.0.0) we aim to provide backward compatibility for the already deployed Flink custom resources (`FlinkDeployment` ,`FlinkSessionJob`).
+
+This means that if you have Flink resources deployed (and Flink applications running), you can still safely upgrade to newer versions of the operator and CRD without any problems.
+This should ensure a smooth operational experience where you can always benefit from the latest fixes and improvements without risking current deployments.
+
+{{< hint warning >}}
+We do not guarantee that you will always be able to deploy new resources using old API versions.
+For example when in the future we upgrade from `v1beta1 -> v1`, while it is guaranteed that your current jobs won't fail, you might need to upgrade to `v1` before you can submit new resources.
+{{< /hint >}}
+
+## Contents of the resource status
+
+Currently, the `FlinkDeployment` and `FlinkSessionJob` resources contain a very detailed status information that contains everything necessary for the operator to manage the resources.
+Most of this information should be considered ***internal*** to operator logic and is subject to change/disappear in the future.
+
+In general, we are aiming to slim down the status to only contain information that is relevant to users/client for the next major CRD version (`v1`).
+
+## Java API compatibility
+
+While it is very important for us to provide [backward compatibility for the custom resources](#custom-resource-backward-compatibility), at this time we cannot provide the same guarantee for Java API.
+
+The operator related java interfaces, such as Validators should be considered experimental/evolving and can possibly break between releases.
+
+## Helm chart compatibility
+
+The Helm chart included in the operator repo provides a convenient way to deploy and manage the operator in complex environments.
+We are constantly working on improving the configuration parameters and features to cover all the different scenarios.
+
+At this time we do not provide backward compatibility for the Helm chart configuration / components. Please verify your custom settings before upgrading to a new version.
diff --git a/docs/content/docs/operations/upgrade.md b/docs/content/docs/operations/upgrade.md
index 1b976ce..815d574 100644
--- a/docs/content/docs/operations/upgrade.md
+++ b/docs/content/docs/operations/upgrade.md
@@ -1,5 +1,5 @@
---
-title: "Upgrade"
+title: "Operator Upgrades"
weight: 4
type: docs
aliases:
@@ -24,8 +24,68 @@ specific language governing permissions and limitations
under the License.
-->
-# Upgrade the CRD
-During development, we delete the old CRD and create it from scratch. In an environment with existing deployments that is not possible, as deleting the CRD would wipe out all existing CRs with the `kind: FlinkDeployment`.
+# Operator Upgrade Process
+
+This page details the process of upgrading the operator to a new version.
+
+Please check the [compatibility page]({{< ref "docs/operations/compatibility" >}}) for the complete overview of the backward compatibility guarantees before upgrading to new versions.
+
+{{< hint danger >}}
+Upgrading from the preview/experimental `v1alpha1` release to `v1beta1` requires a one time manual process.
+Please check the [related section](#upgrading-from-v1alpha1---v1beta1).
+{{< /hint >}}
+
+## Normal Upgrade Process
+
+Normally upgrading the operator to a new release or development version consists of the following two steps:
+1. Upgrading the CRDs
+2. Upgrading the Helm deployment
+
+We will cover these steps in detail in the next sections.
+
+### 1. Upgrading the CRD
+
+The first step of the upgrade process is upgrading the CRDs for `FlinkDeployment` and `FlinkSessionJob` resources.
+This step must be completed manually and is not part of the helm installation logic.
+
+```sh
+kubectl replace -f helm/flink-kubernetes-operator/crds/flinkdeployments.flink.apache.org-v1.yml
+kubectl replace -f helm/flink-kubernetes-operator/crds/flinksessionjobs.flink.apache.org-v1.yml
+```
+
+{{< hint danger >}}
+Please note that we are using the `replace` command here which ensures that running deployments are unaffected.
+{{< /hint >}}
+
+### 2. Upgrading the Helm deployment
+
+Once we have the new CRDs versions we can upgrade the Helm deployment.
+
+```sh
+# Uninstall running Helm deployment
+helm uninstall flink-kubernetes-operator
+helm install ...
+```
+
+The exact installation command depends on your current environment and settings. Please see the [helm page]({{< ref "docs/operations/helm" >}}) for details.
+
+## Upgrading from v1alpha1 -> v1beta1
+
+The first stable `v1beta1` release introduced some breaking changes on the operator side when upgrading from the preview (`v1alpha1`) release.
+These changes require a one time manual upgrade process for the running jobs.
+
+### Upgrading without existing FlinkDeployments
+
+In an environment without any `FlinkDeployments` you simply need to uninstall the operator and delete the v1alpha1 CRD.
+
+```sh
+helm uninstall flink-kubernetes-operator
+kubectl delete crd flinkdeployments.flink.apache.org
+# Now simply reinstall the operator with the new v1beta1 version
+```
+
+### Upgrading with existing FlinkDeployments
+
The following steps demonstrate the CRD upgrade process from `v1alpha1` to `v1beta1` in an environment with an existing [stateful](https://github.com/apache/flink-kubernetes-operator/blob/main/examples/basic-checkpoint-ha.yaml) job with an old `v1alpha1` apiVersion. After the CRD upgrade, the job will resumed from the savepoint.
1. Suspend the job and create savepoint: