You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by GitBox <gi...@apache.org> on 2022/04/18 21:44:28 UTC
[GitHub] [incubator-seatunnel] GezimSejdiu opened a new pull request, #1712: [doc] Add guide on how to Set Up SeaTunnel with Kubernetes
GezimSejdiu opened a new pull request, #1712:
URL: https://github.com/apache/incubator-seatunnel/pull/1712
<!--
Thank you for contributing to SeaTunnel! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
## Contribution Checklist
- Make sure that the pull request corresponds to a [GITHUB issue](https://github.com/apache/incubator-seatunnel/issues).
- Name the pull request in the form "[Feature] [component] Title of the pull request", where *Feature* can be replaced by `Hotfix`, `Bug`, etc.
- Minor fixes should be named following this pattern: `[hotfix] [docs] Fix typo in README.md doc`.
-->
## Purpose of this pull request
Hey SeaTunnel community,
This pull request provides a guide (as of now, Flink only) on using/setting up SeaTunnel via Kubernetes.
It is a great starting point to also
<!-- Describe the purpose of this pull request. For example: This pull request adds checkstyle plugin.-->
## Check list
* [X] Code changed are covered with tests, or it does not need tests for reason: `doc`
* [ ] If any new Jar binary package adding in your PR, please add License Notice according
[New License Guide](https://github.com/apache/incubator-seatunnel/blob/dev/docs/en/contribution/new-license.md)
* [X] If necessary, please update the documentation to describe the new feature. https://github.com/apache/incubator-seatunnel/tree/dev/docs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] zhongjiajie commented on a diff in pull request #1712: [doc] Add guide on how to Set Up SeaTunnel with Kubernetes
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on code in PR #1712:
URL: https://github.com/apache/incubator-seatunnel/pull/1712#discussion_r852540868
##########
docs/en/start/kubernetes.mdx:
##########
@@ -0,0 +1,269 @@
+---
+sidebar_position: 4
+---
+
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Set Up with Kubernetes
+
+This section provides a quick guide to using SeaTunnel with Kubernetes.
+
+## Prerequisites
+
+We assume that you have a local installations of the following:
+
+- [docker](https://docs.docker.com/)
+- [kubernetes](https://kubernetes.io/)
+- [helm](https://helm.sh/docs/intro/quickstart/)
+
+So that the `kubectl` and `helm` commands are available on your local system.
+
+For kubernetes [minikube](https://minikube.sigs.k8s.io/docs/start/) is our choice, at the time of writing this we are using version v1.23.3. You can start a cluster with the following command:
+
+```bash
+minikube start --kubernetes-version=v1.23.3
+```
+
+## Installation
+
+### SeaTunnel docker image
+
+To run the image with SeaTunnel, first create a `Dockerfile`:
+
+<Tabs
+ groupId="engine-type"
+ defaultValue="flink"
+ values={[
+ {label: 'Flink', value: 'flink'},
+ ]}>
+<TabItem value="flink">
+
+```shell
+FROM flink:1.13
+
+ENV SEATUNNEL_VERSION="2.1.0"
+
+RUN wget https://archive.apache.org/dist/incubator/seatunnel/${SEATUNNEL_VERSION}/apache-seatunnel-incubating-${SEATUNNEL_VERSION}-bin.tar.gz
+RUN tar -xzvf apache-seatunnel-incubating-${SEATUNNEL_VERSION}-bin.tar.gz
+
+RUN mkdir -p $FLINK_HOME/usrlib
+RUN cp apache-seatunnel-incubating-${SEATUNNEL_VERSION}/lib/seatunnel-core-flink.jar $FLINK_HOME/usrlib/seatunnel-core-flink.jar
+
+RUN rm -fr apache-seatunnel-incubating-${SEATUNNEL_VERSION}*
+```
+
+Then run the following commands to build the image:
+```bash
+docker build -t seatunnel:2.1.0-flink-1.13 -f Dockerfile .
+```
+Image `seatunnel:2.1.0-flink-1.13` need to be present in the host (minikube) so that the deployment can take place.
+
+Load image to minikube via:
+```bash
+minikube image load seatunnel:2.1.0-flink-1.13
+```
+
+</TabItem>
+</Tabs>
+
+### Deploying the operator
+
+<Tabs
+ groupId="engine-type"
+ defaultValue="flink"
+ values={[
+ {label: 'Flink', value: 'flink'},
+ ]}>
+<TabItem value="flink">
+
+The steps below provide a quick walk-through on setting up the Flink Kubernetes Operator.
+
+Install the certificate manager on your Kubernetes cluster to enable adding the webhook component (only needed once per Kubernetes cluster):
+
+```bash
+kubectl create -f https://github.com/jetstack/cert-manager/releases/download/v1.7.1/cert-manager.yaml
+```
+Now you can deploy the latest stable Flink Kubernetes Operator version using the included Helm chart:
+
+```bash
+
+helm repo add flink-operator-repo https://downloads.apache.org/flink/flink-kubernetes-operator-0.1.0/
+
+helm install flink-kubernetes-operator flink-operator-repo/flink-kubernetes-operator
+```
+
+You may verify your installation via `kubectl`:
+
+```bash
+➜ seatunnel git:(main) ✗ kubectl get pods
Review Comment:
We should use native bash instead of custom like `zsh` or `fish`
```suggestion
$ kubectl get pods
```
##########
docs/en/start/kubernetes.mdx:
##########
@@ -0,0 +1,269 @@
+---
+sidebar_position: 4
+---
+
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Set Up with Kubernetes
+
+This section provides a quick guide to using SeaTunnel with Kubernetes.
+
+## Prerequisites
+
+We assume that you have a local installations of the following:
+
+- [docker](https://docs.docker.com/)
+- [kubernetes](https://kubernetes.io/)
+- [helm](https://helm.sh/docs/intro/quickstart/)
+
+So that the `kubectl` and `helm` commands are available on your local system.
+
+For kubernetes [minikube](https://minikube.sigs.k8s.io/docs/start/) is our choice, at the time of writing this we are using version v1.23.3. You can start a cluster with the following command:
+
+```bash
+minikube start --kubernetes-version=v1.23.3
+```
+
+## Installation
+
+### SeaTunnel docker image
+
+To run the image with SeaTunnel, first create a `Dockerfile`:
+
+<Tabs
+ groupId="engine-type"
+ defaultValue="flink"
+ values={[
+ {label: 'Flink', value: 'flink'},
+ ]}>
+<TabItem value="flink">
+
+```shell
Review Comment:
I think syntax `Dockerfile` is support now, am I right?
```suggestion
```Dockerfile
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] zhongjiajie commented on pull request #1712: [doc] Add guide on how to Set Up SeaTunnel with Kubernetes
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on PR #1712:
URL: https://github.com/apache/incubator-seatunnel/pull/1712#issuecomment-1102026972
> Maybe you need to add an item in https://github.com/apache/incubator-seatunnel/blob/dev/docs/sidebars.js pointing to this new file?
good catch! we should add it to https://github.com/apache/incubator-seatunnel/blob/dev/docs/sidebars.js#L71
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] GezimSejdiu commented on a diff in pull request #1712: [doc] Add guide on how to Set Up SeaTunnel with Kubernetes
Posted by GitBox <gi...@apache.org>.
GezimSejdiu commented on code in PR #1712:
URL: https://github.com/apache/incubator-seatunnel/pull/1712#discussion_r853453710
##########
docs/en/start/kubernetes.mdx:
##########
@@ -0,0 +1,269 @@
+---
+sidebar_position: 4
+---
+
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Set Up with Kubernetes
+
+This section provides a quick guide to using SeaTunnel with Kubernetes.
+
+## Prerequisites
+
+We assume that you have a local installations of the following:
+
+- [docker](https://docs.docker.com/)
+- [kubernetes](https://kubernetes.io/)
+- [helm](https://helm.sh/docs/intro/quickstart/)
+
+So that the `kubectl` and `helm` commands are available on your local system.
+
+For kubernetes [minikube](https://minikube.sigs.k8s.io/docs/start/) is our choice, at the time of writing this we are using version v1.23.3. You can start a cluster with the following command:
+
+```bash
+minikube start --kubernetes-version=v1.23.3
+```
+
+## Installation
+
+### SeaTunnel docker image
+
+To run the image with SeaTunnel, first create a `Dockerfile`:
+
+<Tabs
+ groupId="engine-type"
+ defaultValue="flink"
+ values={[
+ {label: 'Flink', value: 'flink'},
+ ]}>
+<TabItem value="flink">
+
+```shell
Review Comment:
Good catch. I just update to using `Dockerfile` syntax. Thanks for pointing that out.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] CalvinKirs commented on pull request #1712: [doc] Add guide on how to Set Up SeaTunnel with Kubernetes
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on PR #1712:
URL: https://github.com/apache/incubator-seatunnel/pull/1712#issuecomment-1103408502
Well done:)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] GezimSejdiu commented on pull request #1712: [doc] Add guide on how to Set Up SeaTunnel with Kubernetes
Posted by GitBox <gi...@apache.org>.
GezimSejdiu commented on PR #1712:
URL: https://github.com/apache/incubator-seatunnel/pull/1712#issuecomment-1103099242
> > Maybe you need to add an item in https://github.com/apache/incubator-seatunnel/blob/dev/docs/sidebars.js pointing to this new file?
>
> good catch! we should add it to https://github.com/apache/incubator-seatunnel/blob/dev/docs/sidebars.js#L71
I also did add it to the sidebar.js -- wasn't aware of that since I thought (checked on the file) that it can also be generated automatically.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] zhongjiajie commented on a diff in pull request #1712: [doc] Add guide on how to Set Up SeaTunnel with Kubernetes
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on code in PR #1712:
URL: https://github.com/apache/incubator-seatunnel/pull/1712#discussion_r852540868
##########
docs/en/start/kubernetes.mdx:
##########
@@ -0,0 +1,269 @@
+---
+sidebar_position: 4
+---
+
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Set Up with Kubernetes
+
+This section provides a quick guide to using SeaTunnel with Kubernetes.
+
+## Prerequisites
+
+We assume that you have a local installations of the following:
+
+- [docker](https://docs.docker.com/)
+- [kubernetes](https://kubernetes.io/)
+- [helm](https://helm.sh/docs/intro/quickstart/)
+
+So that the `kubectl` and `helm` commands are available on your local system.
+
+For kubernetes [minikube](https://minikube.sigs.k8s.io/docs/start/) is our choice, at the time of writing this we are using version v1.23.3. You can start a cluster with the following command:
+
+```bash
+minikube start --kubernetes-version=v1.23.3
+```
+
+## Installation
+
+### SeaTunnel docker image
+
+To run the image with SeaTunnel, first create a `Dockerfile`:
+
+<Tabs
+ groupId="engine-type"
+ defaultValue="flink"
+ values={[
+ {label: 'Flink', value: 'flink'},
+ ]}>
+<TabItem value="flink">
+
+```shell
+FROM flink:1.13
+
+ENV SEATUNNEL_VERSION="2.1.0"
+
+RUN wget https://archive.apache.org/dist/incubator/seatunnel/${SEATUNNEL_VERSION}/apache-seatunnel-incubating-${SEATUNNEL_VERSION}-bin.tar.gz
+RUN tar -xzvf apache-seatunnel-incubating-${SEATUNNEL_VERSION}-bin.tar.gz
+
+RUN mkdir -p $FLINK_HOME/usrlib
+RUN cp apache-seatunnel-incubating-${SEATUNNEL_VERSION}/lib/seatunnel-core-flink.jar $FLINK_HOME/usrlib/seatunnel-core-flink.jar
+
+RUN rm -fr apache-seatunnel-incubating-${SEATUNNEL_VERSION}*
+```
+
+Then run the following commands to build the image:
+```bash
+docker build -t seatunnel:2.1.0-flink-1.13 -f Dockerfile .
+```
+Image `seatunnel:2.1.0-flink-1.13` need to be present in the host (minikube) so that the deployment can take place.
+
+Load image to minikube via:
+```bash
+minikube image load seatunnel:2.1.0-flink-1.13
+```
+
+</TabItem>
+</Tabs>
+
+### Deploying the operator
+
+<Tabs
+ groupId="engine-type"
+ defaultValue="flink"
+ values={[
+ {label: 'Flink', value: 'flink'},
+ ]}>
+<TabItem value="flink">
+
+The steps below provide a quick walk-through on setting up the Flink Kubernetes Operator.
+
+Install the certificate manager on your Kubernetes cluster to enable adding the webhook component (only needed once per Kubernetes cluster):
+
+```bash
+kubectl create -f https://github.com/jetstack/cert-manager/releases/download/v1.7.1/cert-manager.yaml
+```
+Now you can deploy the latest stable Flink Kubernetes Operator version using the included Helm chart:
+
+```bash
+
+helm repo add flink-operator-repo https://downloads.apache.org/flink/flink-kubernetes-operator-0.1.0/
+
+helm install flink-kubernetes-operator flink-operator-repo/flink-kubernetes-operator
+```
+
+You may verify your installation via `kubectl`:
+
+```bash
+➜ seatunnel git:(main) ✗ kubectl get pods
Review Comment:
We should use native bash instead of custom like `zsh` or `fish`
```suggestion
kubectl get pods
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] CalvinKirs merged pull request #1712: [doc] Add guide on how to Set Up SeaTunnel with Kubernetes
Posted by GitBox <gi...@apache.org>.
CalvinKirs merged PR #1712:
URL: https://github.com/apache/incubator-seatunnel/pull/1712
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] zhongjiajie commented on pull request #1712: [doc] Add guide on how to Set Up SeaTunnel with Kubernetes
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on PR #1712:
URL: https://github.com/apache/incubator-seatunnel/pull/1712#issuecomment-1103439633
It could be auto-generated in some cases,
https://github.com/apache/incubator-seatunnel/blob/dev/docs/sidebars.js#L90-L94
but when we want content in a specific order instead of alphabetically, we should point them manually
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] GezimSejdiu commented on a diff in pull request #1712: [doc] Add guide on how to Set Up SeaTunnel with Kubernetes
Posted by GitBox <gi...@apache.org>.
GezimSejdiu commented on code in PR #1712:
URL: https://github.com/apache/incubator-seatunnel/pull/1712#discussion_r853454230
##########
docs/en/start/kubernetes.mdx:
##########
@@ -0,0 +1,269 @@
+---
+sidebar_position: 4
+---
+
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Set Up with Kubernetes
+
+This section provides a quick guide to using SeaTunnel with Kubernetes.
+
+## Prerequisites
+
+We assume that you have a local installations of the following:
+
+- [docker](https://docs.docker.com/)
+- [kubernetes](https://kubernetes.io/)
+- [helm](https://helm.sh/docs/intro/quickstart/)
+
+So that the `kubectl` and `helm` commands are available on your local system.
+
+For kubernetes [minikube](https://minikube.sigs.k8s.io/docs/start/) is our choice, at the time of writing this we are using version v1.23.3. You can start a cluster with the following command:
+
+```bash
+minikube start --kubernetes-version=v1.23.3
+```
+
+## Installation
+
+### SeaTunnel docker image
+
+To run the image with SeaTunnel, first create a `Dockerfile`:
+
+<Tabs
+ groupId="engine-type"
+ defaultValue="flink"
+ values={[
+ {label: 'Flink', value: 'flink'},
+ ]}>
+<TabItem value="flink">
+
+```shell
+FROM flink:1.13
+
+ENV SEATUNNEL_VERSION="2.1.0"
+
+RUN wget https://archive.apache.org/dist/incubator/seatunnel/${SEATUNNEL_VERSION}/apache-seatunnel-incubating-${SEATUNNEL_VERSION}-bin.tar.gz
+RUN tar -xzvf apache-seatunnel-incubating-${SEATUNNEL_VERSION}-bin.tar.gz
+
+RUN mkdir -p $FLINK_HOME/usrlib
+RUN cp apache-seatunnel-incubating-${SEATUNNEL_VERSION}/lib/seatunnel-core-flink.jar $FLINK_HOME/usrlib/seatunnel-core-flink.jar
+
+RUN rm -fr apache-seatunnel-incubating-${SEATUNNEL_VERSION}*
+```
+
+Then run the following commands to build the image:
+```bash
+docker build -t seatunnel:2.1.0-flink-1.13 -f Dockerfile .
+```
+Image `seatunnel:2.1.0-flink-1.13` need to be present in the host (minikube) so that the deployment can take place.
+
+Load image to minikube via:
+```bash
+minikube image load seatunnel:2.1.0-flink-1.13
+```
+
+</TabItem>
+</Tabs>
+
+### Deploying the operator
+
+<Tabs
+ groupId="engine-type"
+ defaultValue="flink"
+ values={[
+ {label: 'Flink', value: 'flink'},
+ ]}>
+<TabItem value="flink">
+
+The steps below provide a quick walk-through on setting up the Flink Kubernetes Operator.
+
+Install the certificate manager on your Kubernetes cluster to enable adding the webhook component (only needed once per Kubernetes cluster):
+
+```bash
+kubectl create -f https://github.com/jetstack/cert-manager/releases/download/v1.7.1/cert-manager.yaml
+```
+Now you can deploy the latest stable Flink Kubernetes Operator version using the included Helm chart:
+
+```bash
+
+helm repo add flink-operator-repo https://downloads.apache.org/flink/flink-kubernetes-operator-0.1.0/
+
+helm install flink-kubernetes-operator flink-operator-repo/flink-kubernetes-operator
+```
+
+You may verify your installation via `kubectl`:
+
+```bash
+➜ seatunnel git:(main) ✗ kubectl get pods
Review Comment:
Can't agree more. Just update to using `bash` only.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org