You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2022/04/04 19:59:25 UTC
[GitHub] [flink-kubernetes-operator] morhidi opened a new pull request, #155: [FLINK-26663] Pod augmentation for the operator
morhidi opened a new pull request, #155:
URL: https://github.com/apache/flink-kubernetes-operator/pull/155
- adding basic example and docs for pod augmentation using Helm + kustomize
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-kubernetes-operator] morhidi commented on pull request #155: [FLINK-26663] Pod augmentation for the operator
Posted by GitBox <gi...@apache.org>.
morhidi commented on PR #155:
URL: https://github.com/apache/flink-kubernetes-operator/pull/155#issuecomment-1087956079
cc @wangyang0918 @gyfora @mbalassi
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-kubernetes-operator] morhidi commented on a diff in pull request #155: [FLINK-26663] Pod augmentation for the operator
Posted by GitBox <gi...@apache.org>.
morhidi commented on code in PR #155:
URL: https://github.com/apache/flink-kubernetes-operator/pull/155#discussion_r844924937
##########
docs/content/docs/operations/helm.md:
##########
@@ -109,3 +109,107 @@ spec:
skipCrds: true
...
```
+
+## Advanced customization techniques
+The Helm chart does not aim to provide configuration options for all the possible deployment scenarios of the Operator. There are use cases for injecting common tools and/or sidecars in most enterprise environments that simply cannot be covered by public Helm charts.
+
+Fortunately, [post rendering](https://helm.sh/docs/topics/advanced/#post-rendering) in Helm gives you the ability to manually manipulate manifests before they are installed on a Kubernetes cluster. This allows users to use tools like [kustomize](https://kustomize.io) to apply configuration changes without the need to fork public charts.
+
+The GitHub repository for the Operator contains a simple [example](https://github.com/apache/flink-kubernetes-operator/tree/main/examples/kustomize) on how to augment the Operator Deployment with a [fluent-bit](https://docs.fluentbit.io/manual) sidecar container and adjust container resources using `kustomize`.
+
+The example demonstrates that we can still use a `values.yaml` file to override the default Helm values for changing the log configuration, for example:
+
+```yaml
+
+operatorConfiguration:
+ ...
+ log4j2.properties: |+
+ rootLogger.appenderRef.file.ref = LogFile
+ appender.file.name = LogFile
+ appender.file.type = File
+ appender.file.append = false
+ appender.file.fileName = ${sys:log.file}
+ appender.file.layout.type = PatternLayout
+ appender.file.layout.pattern = %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %-60c %x - %m%n
+
+jvmArgs:
+ webhook: "-Dlog.file=/opt/flink/log/webhook.log -Xms256m -Xmx256m"
+ operator: "-Dlog.file=/opt/flink/log/operator.log -Xms2048m -Xmx2048m"
+```
+
+But we cannot ingest our fluent-bit sidecar for example unless we patch the deployment using `kustomize`
+
+```yaml
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+ name: not-important
+spec:
+ template:
+ spec:
+ containers:
+ - name: flink-kubernetes-operator
+ volumeMounts:
+ - name: flink-log
+ mountPath: /opt/flink/log
+ resources:
+ requests:
+ memory: "2.5Gi"
+ cpu: "1000m"
+ limits:
+ memory: "2.5Gi"
+ cpu: "2000m"
+ - name: flink-webhook
+ volumeMounts:
+ - name: flink-log
+ mountPath: /opt/flink/log
+ resources:
+ requests:
+ memory: "0.5Gi"
+ cpu: "200m"
+ limits:
+ memory: "0.5Gi"
+ cpu: "500m"
+ - name: fluentbit
+ image: fluent/fluent-bit:1.8.12
+ command: [ 'sh','-c','/fluent-bit/bin/fluent-bit -i tail -p path=/opt/flink/log/*.log -p multiline.parser=java -o stdout' ]
Review Comment:
it takes a few seconds, let me adjust the flush interval
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-kubernetes-operator] morhidi commented on a diff in pull request #155: [FLINK-26663] Pod augmentation for the operator
Posted by GitBox <gi...@apache.org>.
morhidi commented on code in PR #155:
URL: https://github.com/apache/flink-kubernetes-operator/pull/155#discussion_r843792975
##########
docs/content/docs/operations/helm.md:
##########
@@ -109,3 +109,24 @@ spec:
skipCrds: true
...
```
+
+## Advanced customization techniques
+The Helm chart does not aim to provide configuration options for all the possible deployment scenarios of the Operator. There are use cases for injecting common tools and/or sidecars in most enterprise environments that simply cannot be covered by public Helm charts.
+
+Fortunately, [post rendering](https://helm.sh/docs/topics/advanced/#post-rendering) in Helm gives you the ability to manually manipulate manifests before they are installed on a Kubernetes cluster. This allows users to use tools like [kustomize](https://kustomize.io) to apply configuration changes without the need to fork public charts.
+
+The GitHub repository for the Operator contains a simple [example](https://github.com/apache/flink-kubernetes-operator/tree/main/examples/kustomize) on how to augment the Operator Deployment with an init container using `kustomize`.
+
+```yaml
+initContainers:
+ - name: busybox
+ image: busybox:latest
+ command: [ 'sh','-c','echo Happy Customization!' ]```
+```
+
+You can enable the example using the `--post-renderer` option when installing the Operator with Helm:
+
+```yaml
Review Comment:
fixed, thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-kubernetes-operator] morhidi commented on pull request #155: [FLINK-26663] Pod augmentation for the operator
Posted by GitBox <gi...@apache.org>.
morhidi commented on PR #155:
URL: https://github.com/apache/flink-kubernetes-operator/pull/155#issuecomment-1090128136
> We need to clarify some behaviors.
>
> * What's the priority of `--set` and `kustomize`?
> * Could we use `kustomize` to decorate some existing container, e.g. adding resources for flink-kubernetes-operator or flink-webhook? If it is true, then it will be great if we could have an example for setting the resources and combined with `jvmArgs`.
Thanks @wangyang0918 added a more sophisticated example that combines the Helm values with customize. PTAL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-kubernetes-operator] gyfora merged pull request #155: [FLINK-26663] Pod augmentation for the operator
Posted by GitBox <gi...@apache.org>.
gyfora merged PR #155:
URL: https://github.com/apache/flink-kubernetes-operator/pull/155
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-kubernetes-operator] wangyang0918 commented on a diff in pull request #155: [FLINK-26663] Pod augmentation for the operator
Posted by GitBox <gi...@apache.org>.
wangyang0918 commented on code in PR #155:
URL: https://github.com/apache/flink-kubernetes-operator/pull/155#discussion_r844802012
##########
docs/content/docs/operations/helm.md:
##########
@@ -109,3 +109,107 @@ spec:
skipCrds: true
...
```
+
+## Advanced customization techniques
+The Helm chart does not aim to provide configuration options for all the possible deployment scenarios of the Operator. There are use cases for injecting common tools and/or sidecars in most enterprise environments that simply cannot be covered by public Helm charts.
+
+Fortunately, [post rendering](https://helm.sh/docs/topics/advanced/#post-rendering) in Helm gives you the ability to manually manipulate manifests before they are installed on a Kubernetes cluster. This allows users to use tools like [kustomize](https://kustomize.io) to apply configuration changes without the need to fork public charts.
+
+The GitHub repository for the Operator contains a simple [example](https://github.com/apache/flink-kubernetes-operator/tree/main/examples/kustomize) on how to augment the Operator Deployment with a [fluent-bit](https://docs.fluentbit.io/manual) sidecar container and adjust container resources using `kustomize`.
+
+The example demonstrates that we can still use a `values.yaml` file to override the default Helm values for changing the log configuration, for example:
+
+```yaml
+
+operatorConfiguration:
+ ...
+ log4j2.properties: |+
+ rootLogger.appenderRef.file.ref = LogFile
+ appender.file.name = LogFile
+ appender.file.type = File
+ appender.file.append = false
+ appender.file.fileName = ${sys:log.file}
+ appender.file.layout.type = PatternLayout
+ appender.file.layout.pattern = %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %-60c %x - %m%n
+
+jvmArgs:
+ webhook: "-Dlog.file=/opt/flink/log/webhook.log -Xms256m -Xmx256m"
+ operator: "-Dlog.file=/opt/flink/log/operator.log -Xms2048m -Xmx2048m"
+```
+
+But we cannot ingest our fluent-bit sidecar for example unless we patch the deployment using `kustomize`
+
+```yaml
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+ name: not-important
+spec:
+ template:
+ spec:
+ containers:
+ - name: flink-kubernetes-operator
+ volumeMounts:
+ - name: flink-log
+ mountPath: /opt/flink/log
+ resources:
+ requests:
+ memory: "2.5Gi"
+ cpu: "1000m"
+ limits:
+ memory: "2.5Gi"
+ cpu: "2000m"
+ - name: flink-webhook
+ volumeMounts:
+ - name: flink-log
+ mountPath: /opt/flink/log
+ resources:
+ requests:
+ memory: "0.5Gi"
+ cpu: "200m"
+ limits:
+ memory: "0.5Gi"
+ cpu: "500m"
+ - name: fluentbit
+ image: fluent/fluent-bit:1.8.12
+ command: [ 'sh','-c','/fluent-bit/bin/fluent-bit -i tail -p path=/opt/flink/log/*.log -p multiline.parser=java -o stdout' ]
Review Comment:
It is strange. I will have a try whether debug version could work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-kubernetes-operator] morhidi commented on pull request #155: [FLINK-26663] Pod augmentation for the operator
Posted by GitBox <gi...@apache.org>.
morhidi commented on PR #155:
URL: https://github.com/apache/flink-kubernetes-operator/pull/155#issuecomment-1090559785
> Actually, what I mean is to configure the memory/cpu resources, which could cover [FLINK-27001](https://issues.apache.org/jira/browse/FLINK-27001). Anyway, we could still do this in [FLINK-27001](https://issues.apache.org/jira/browse/FLINK-27001) later.
ah ok, added those to the example too, cc @Aitozi
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-kubernetes-operator] wangyang0918 commented on a diff in pull request #155: [FLINK-26663] Pod augmentation for the operator
Posted by GitBox <gi...@apache.org>.
wangyang0918 commented on code in PR #155:
URL: https://github.com/apache/flink-kubernetes-operator/pull/155#discussion_r844856310
##########
docs/content/docs/operations/helm.md:
##########
@@ -109,3 +109,107 @@ spec:
skipCrds: true
...
```
+
+## Advanced customization techniques
+The Helm chart does not aim to provide configuration options for all the possible deployment scenarios of the Operator. There are use cases for injecting common tools and/or sidecars in most enterprise environments that simply cannot be covered by public Helm charts.
+
+Fortunately, [post rendering](https://helm.sh/docs/topics/advanced/#post-rendering) in Helm gives you the ability to manually manipulate manifests before they are installed on a Kubernetes cluster. This allows users to use tools like [kustomize](https://kustomize.io) to apply configuration changes without the need to fork public charts.
+
+The GitHub repository for the Operator contains a simple [example](https://github.com/apache/flink-kubernetes-operator/tree/main/examples/kustomize) on how to augment the Operator Deployment with a [fluent-bit](https://docs.fluentbit.io/manual) sidecar container and adjust container resources using `kustomize`.
+
+The example demonstrates that we can still use a `values.yaml` file to override the default Helm values for changing the log configuration, for example:
+
+```yaml
+
+operatorConfiguration:
+ ...
+ log4j2.properties: |+
+ rootLogger.appenderRef.file.ref = LogFile
+ appender.file.name = LogFile
+ appender.file.type = File
+ appender.file.append = false
+ appender.file.fileName = ${sys:log.file}
+ appender.file.layout.type = PatternLayout
+ appender.file.layout.pattern = %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %-60c %x - %m%n
+
+jvmArgs:
+ webhook: "-Dlog.file=/opt/flink/log/webhook.log -Xms256m -Xmx256m"
+ operator: "-Dlog.file=/opt/flink/log/operator.log -Xms2048m -Xmx2048m"
+```
+
+But we cannot ingest our fluent-bit sidecar for example unless we patch the deployment using `kustomize`
+
+```yaml
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+ name: not-important
+spec:
+ template:
+ spec:
+ containers:
+ - name: flink-kubernetes-operator
+ volumeMounts:
+ - name: flink-log
+ mountPath: /opt/flink/log
+ resources:
+ requests:
+ memory: "2.5Gi"
+ cpu: "1000m"
+ limits:
+ memory: "2.5Gi"
+ cpu: "2000m"
+ - name: flink-webhook
+ volumeMounts:
+ - name: flink-log
+ mountPath: /opt/flink/log
+ resources:
+ requests:
+ memory: "0.5Gi"
+ cpu: "200m"
+ limits:
+ memory: "0.5Gi"
+ cpu: "500m"
+ - name: fluentbit
+ image: fluent/fluent-bit:1.8.12
+ command: [ 'sh','-c','/fluent-bit/bin/fluent-bit -i tail -p path=/opt/flink/log/*.log -p multiline.parser=java -o stdout' ]
Review Comment:
The flink-kubernetes-operator could run normally now. But the `fluent` does not tail the logs correctly. Do I miss some thing?
```
wangyang-pc:flink-kubernetes-operator danrtsey.wy$ kubectl logs flink-kubernetes-operator-7c6b75b9bf-nrg9x -c fluentbit
Fluent Bit v1.8.12
* Copyright (C) 2019-2021 The Fluent Bit Authors
* Copyright (C) 2015-2018 Treasure Data
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io
[2022/04/07 07:54:58] [ info] [engine] started (pid=8)
[2022/04/07 07:54:58] [ info] [storage] version=1.1.5, initializing...
[2022/04/07 07:54:58] [ info] [storage] in-memory
[2022/04/07 07:54:58] [ info] [storage] normal synchronization mode, checksum disabled, max_chunks_up=128
[2022/04/07 07:54:58] [ info] [cmetrics] version=0.2.2
[2022/04/07 07:54:58] [ info] [input:tail:tail.0] multiline core started
[2022/04/07 07:54:58] [ info] [sp] stream processor started
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-kubernetes-operator] wangyang0918 commented on a diff in pull request #155: [FLINK-26663] Pod augmentation for the operator
Posted by GitBox <gi...@apache.org>.
wangyang0918 commented on code in PR #155:
URL: https://github.com/apache/flink-kubernetes-operator/pull/155#discussion_r844937962
##########
docs/content/docs/operations/helm.md:
##########
@@ -109,3 +109,107 @@ spec:
skipCrds: true
...
```
+
+## Advanced customization techniques
+The Helm chart does not aim to provide configuration options for all the possible deployment scenarios of the Operator. There are use cases for injecting common tools and/or sidecars in most enterprise environments that simply cannot be covered by public Helm charts.
+
+Fortunately, [post rendering](https://helm.sh/docs/topics/advanced/#post-rendering) in Helm gives you the ability to manually manipulate manifests before they are installed on a Kubernetes cluster. This allows users to use tools like [kustomize](https://kustomize.io) to apply configuration changes without the need to fork public charts.
+
+The GitHub repository for the Operator contains a simple [example](https://github.com/apache/flink-kubernetes-operator/tree/main/examples/kustomize) on how to augment the Operator Deployment with a [fluent-bit](https://docs.fluentbit.io/manual) sidecar container and adjust container resources using `kustomize`.
+
+The example demonstrates that we can still use a `values.yaml` file to override the default Helm values for changing the log configuration, for example:
+
+```yaml
+
+operatorConfiguration:
+ ...
+ log4j2.properties: |+
+ rootLogger.appenderRef.file.ref = LogFile
+ appender.file.name = LogFile
+ appender.file.type = File
+ appender.file.append = false
+ appender.file.fileName = ${sys:log.file}
+ appender.file.layout.type = PatternLayout
+ appender.file.layout.pattern = %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %-60c %x - %m%n
+
+jvmArgs:
+ webhook: "-Dlog.file=/opt/flink/log/webhook.log -Xms256m -Xmx256m"
+ operator: "-Dlog.file=/opt/flink/log/operator.log -Xms2048m -Xmx2048m"
+```
+
+But we cannot ingest our fluent-bit sidecar for example unless we patch the deployment using `kustomize`
+
+```yaml
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+ name: not-important
+spec:
+ template:
+ spec:
+ containers:
+ - name: flink-kubernetes-operator
+ volumeMounts:
+ - name: flink-log
+ mountPath: /opt/flink/log
+ resources:
+ requests:
+ memory: "2.5Gi"
+ cpu: "1000m"
+ limits:
+ memory: "2.5Gi"
+ cpu: "2000m"
+ - name: flink-webhook
+ volumeMounts:
+ - name: flink-log
+ mountPath: /opt/flink/log
+ resources:
+ requests:
+ memory: "0.5Gi"
+ cpu: "200m"
+ limits:
+ memory: "0.5Gi"
+ cpu: "500m"
+ - name: fluentbit
+ image: fluent/fluent-bit:1.8.12
+ command: [ 'sh','-c','/fluent-bit/bin/fluent-bit -i tail -p path=/opt/flink/log/*.log -p multiline.parser=java -o stdout' ]
Review Comment:
Could you share me some light why the `kustomize/values.yaml` does not take effect in my environment? Both the logging and the jvm args does not work for me.
```
helm install flink-kubernetes-operator helm/flink-kubernetes-operator --post-renderer examples/kustomize/render
```
BTW, the changes in the `kustomize/patch.yaml` could take effect.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-kubernetes-operator] wangyang0918 commented on a diff in pull request #155: [FLINK-26663] Pod augmentation for the operator
Posted by GitBox <gi...@apache.org>.
wangyang0918 commented on code in PR #155:
URL: https://github.com/apache/flink-kubernetes-operator/pull/155#discussion_r844962009
##########
docs/content/docs/operations/helm.md:
##########
@@ -109,3 +109,107 @@ spec:
skipCrds: true
...
```
+
+## Advanced customization techniques
+The Helm chart does not aim to provide configuration options for all the possible deployment scenarios of the Operator. There are use cases for injecting common tools and/or sidecars in most enterprise environments that simply cannot be covered by public Helm charts.
+
+Fortunately, [post rendering](https://helm.sh/docs/topics/advanced/#post-rendering) in Helm gives you the ability to manually manipulate manifests before they are installed on a Kubernetes cluster. This allows users to use tools like [kustomize](https://kustomize.io) to apply configuration changes without the need to fork public charts.
+
+The GitHub repository for the Operator contains a simple [example](https://github.com/apache/flink-kubernetes-operator/tree/main/examples/kustomize) on how to augment the Operator Deployment with a [fluent-bit](https://docs.fluentbit.io/manual) sidecar container and adjust container resources using `kustomize`.
+
+The example demonstrates that we can still use a `values.yaml` file to override the default Helm values for changing the log configuration, for example:
+
+```yaml
+
+operatorConfiguration:
+ ...
+ log4j2.properties: |+
+ rootLogger.appenderRef.file.ref = LogFile
+ appender.file.name = LogFile
+ appender.file.type = File
+ appender.file.append = false
+ appender.file.fileName = ${sys:log.file}
+ appender.file.layout.type = PatternLayout
+ appender.file.layout.pattern = %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %-60c %x - %m%n
+
+jvmArgs:
+ webhook: "-Dlog.file=/opt/flink/log/webhook.log -Xms256m -Xmx256m"
+ operator: "-Dlog.file=/opt/flink/log/operator.log -Xms2048m -Xmx2048m"
+```
+
+But we cannot ingest our fluent-bit sidecar for example unless we patch the deployment using `kustomize`
+
+```yaml
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+ name: not-important
+spec:
+ template:
+ spec:
+ containers:
+ - name: flink-kubernetes-operator
+ volumeMounts:
+ - name: flink-log
+ mountPath: /opt/flink/log
+ resources:
+ requests:
+ memory: "2.5Gi"
+ cpu: "1000m"
+ limits:
+ memory: "2.5Gi"
+ cpu: "2000m"
+ - name: flink-webhook
+ volumeMounts:
+ - name: flink-log
+ mountPath: /opt/flink/log
+ resources:
+ requests:
+ memory: "0.5Gi"
+ cpu: "200m"
+ limits:
+ memory: "0.5Gi"
+ cpu: "500m"
+ - name: fluentbit
+ image: fluent/fluent-bit:1.8.12
+ command: [ 'sh','-c','/fluent-bit/bin/fluent-bit -i tail -p path=/opt/flink/log/*.log -p multiline.parser=java -o stdout' ]
Review Comment:
Aha. So `examples/kustomize/values.yaml` will not be merged with `helm/values.yaml`, but just replace. Right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-kubernetes-operator] morhidi commented on a diff in pull request #155: [FLINK-26663] Pod augmentation for the operator
Posted by GitBox <gi...@apache.org>.
morhidi commented on code in PR #155:
URL: https://github.com/apache/flink-kubernetes-operator/pull/155#discussion_r844965707
##########
docs/content/docs/operations/helm.md:
##########
@@ -109,3 +109,107 @@ spec:
skipCrds: true
...
```
+
+## Advanced customization techniques
+The Helm chart does not aim to provide configuration options for all the possible deployment scenarios of the Operator. There are use cases for injecting common tools and/or sidecars in most enterprise environments that simply cannot be covered by public Helm charts.
+
+Fortunately, [post rendering](https://helm.sh/docs/topics/advanced/#post-rendering) in Helm gives you the ability to manually manipulate manifests before they are installed on a Kubernetes cluster. This allows users to use tools like [kustomize](https://kustomize.io) to apply configuration changes without the need to fork public charts.
+
+The GitHub repository for the Operator contains a simple [example](https://github.com/apache/flink-kubernetes-operator/tree/main/examples/kustomize) on how to augment the Operator Deployment with a [fluent-bit](https://docs.fluentbit.io/manual) sidecar container and adjust container resources using `kustomize`.
+
+The example demonstrates that we can still use a `values.yaml` file to override the default Helm values for changing the log configuration, for example:
+
+```yaml
+
+operatorConfiguration:
+ ...
+ log4j2.properties: |+
+ rootLogger.appenderRef.file.ref = LogFile
+ appender.file.name = LogFile
+ appender.file.type = File
+ appender.file.append = false
+ appender.file.fileName = ${sys:log.file}
+ appender.file.layout.type = PatternLayout
+ appender.file.layout.pattern = %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %-60c %x - %m%n
+
+jvmArgs:
+ webhook: "-Dlog.file=/opt/flink/log/webhook.log -Xms256m -Xmx256m"
+ operator: "-Dlog.file=/opt/flink/log/operator.log -Xms2048m -Xmx2048m"
+```
+
+But we cannot ingest our fluent-bit sidecar for example unless we patch the deployment using `kustomize`
+
+```yaml
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+ name: not-important
+spec:
+ template:
+ spec:
+ containers:
+ - name: flink-kubernetes-operator
+ volumeMounts:
+ - name: flink-log
+ mountPath: /opt/flink/log
+ resources:
+ requests:
+ memory: "2.5Gi"
+ cpu: "1000m"
+ limits:
+ memory: "2.5Gi"
+ cpu: "2000m"
+ - name: flink-webhook
+ volumeMounts:
+ - name: flink-log
+ mountPath: /opt/flink/log
+ resources:
+ requests:
+ memory: "0.5Gi"
+ cpu: "200m"
+ limits:
+ memory: "0.5Gi"
+ cpu: "500m"
+ - name: fluentbit
+ image: fluent/fluent-bit:1.8.12
+ command: [ 'sh','-c','/fluent-bit/bin/fluent-bit -i tail -p path=/opt/flink/log/*.log -p multiline.parser=java -o stdout' ]
Review Comment:
correct (actually it's an override in Helm too)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-kubernetes-operator] mbalassi commented on a diff in pull request #155: [FLINK-26663] Pod augmentation for the operator
Posted by GitBox <gi...@apache.org>.
mbalassi commented on code in PR #155:
URL: https://github.com/apache/flink-kubernetes-operator/pull/155#discussion_r843292106
##########
docs/content/docs/operations/helm.md:
##########
@@ -109,3 +109,24 @@ spec:
skipCrds: true
...
```
+
+## Advanced customization techniques
+The Helm chart does not aim to provide configuration options for all the possible deployment scenarios of the Operator. There are use cases for injecting common tools and/or sidecars in most enterprise environments that simply cannot be covered by public Helm charts.
+
+Fortunately, [post rendering](https://helm.sh/docs/topics/advanced/#post-rendering) in Helm gives you the ability to manually manipulate manifests before they are installed on a Kubernetes cluster. This allows users to use tools like [kustomize](https://kustomize.io) to apply configuration changes without the need to fork public charts.
+
+The GitHub repository for the Operator contains a simple [example](https://github.com/apache/flink-kubernetes-operator/tree/main/examples/kustomize) on how to augment the Operator Deployment with an init container using `kustomize`.
+
+```yaml
+initContainers:
+ - name: busybox
+ image: busybox:latest
+ command: [ 'sh','-c','echo Happy Customization!' ]```
+```
+
+You can enable the example using the `--post-renderer` option when installing the Operator with Helm:
+
+```yaml
Review Comment:
nit: shell
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-kubernetes-operator] morhidi commented on a diff in pull request #155: [FLINK-26663] Pod augmentation for the operator
Posted by GitBox <gi...@apache.org>.
morhidi commented on code in PR #155:
URL: https://github.com/apache/flink-kubernetes-operator/pull/155#discussion_r844949045
##########
docs/content/docs/operations/helm.md:
##########
@@ -109,3 +109,107 @@ spec:
skipCrds: true
...
```
+
+## Advanced customization techniques
+The Helm chart does not aim to provide configuration options for all the possible deployment scenarios of the Operator. There are use cases for injecting common tools and/or sidecars in most enterprise environments that simply cannot be covered by public Helm charts.
+
+Fortunately, [post rendering](https://helm.sh/docs/topics/advanced/#post-rendering) in Helm gives you the ability to manually manipulate manifests before they are installed on a Kubernetes cluster. This allows users to use tools like [kustomize](https://kustomize.io) to apply configuration changes without the need to fork public charts.
+
+The GitHub repository for the Operator contains a simple [example](https://github.com/apache/flink-kubernetes-operator/tree/main/examples/kustomize) on how to augment the Operator Deployment with a [fluent-bit](https://docs.fluentbit.io/manual) sidecar container and adjust container resources using `kustomize`.
+
+The example demonstrates that we can still use a `values.yaml` file to override the default Helm values for changing the log configuration, for example:
+
+```yaml
+
+operatorConfiguration:
+ ...
+ log4j2.properties: |+
+ rootLogger.appenderRef.file.ref = LogFile
+ appender.file.name = LogFile
+ appender.file.type = File
+ appender.file.append = false
+ appender.file.fileName = ${sys:log.file}
+ appender.file.layout.type = PatternLayout
+ appender.file.layout.pattern = %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %-60c %x - %m%n
+
+jvmArgs:
+ webhook: "-Dlog.file=/opt/flink/log/webhook.log -Xms256m -Xmx256m"
+ operator: "-Dlog.file=/opt/flink/log/operator.log -Xms2048m -Xmx2048m"
+```
+
+But we cannot ingest our fluent-bit sidecar for example unless we patch the deployment using `kustomize`
+
+```yaml
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+ name: not-important
+spec:
+ template:
+ spec:
+ containers:
+ - name: flink-kubernetes-operator
+ volumeMounts:
+ - name: flink-log
+ mountPath: /opt/flink/log
+ resources:
+ requests:
+ memory: "2.5Gi"
+ cpu: "1000m"
+ limits:
+ memory: "2.5Gi"
+ cpu: "2000m"
+ - name: flink-webhook
+ volumeMounts:
+ - name: flink-log
+ mountPath: /opt/flink/log
+ resources:
+ requests:
+ memory: "0.5Gi"
+ cpu: "200m"
+ limits:
+ memory: "0.5Gi"
+ cpu: "500m"
+ - name: fluentbit
+ image: fluent/fluent-bit:1.8.12
+ command: [ 'sh','-c','/fluent-bit/bin/fluent-bit -i tail -p path=/opt/flink/log/*.log -p multiline.parser=java -o stdout' ]
Review Comment:
you have to reference it with -f values.yaml (it's just still the standard values override) it has noting to do with customise, I just put it to the example folder
```helm install flink-kubernetes-operator helm/flink-kubernetes-operator -f examples/kustomize/values.yaml --post-renderer examples/kustomize/render```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-kubernetes-operator] wangyang0918 commented on pull request #155: [FLINK-26663] Pod augmentation for the operator
Posted by GitBox <gi...@apache.org>.
wangyang0918 commented on PR #155:
URL: https://github.com/apache/flink-kubernetes-operator/pull/155#issuecomment-1090285070
Actually, what I mean is to configure the memory/cpu resources, which could cover FLINK-27001. Anyway, we could still do this in FLINK-27001 later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-kubernetes-operator] wangyang0918 commented on a diff in pull request #155: [FLINK-26663] Pod augmentation for the operator
Posted by GitBox <gi...@apache.org>.
wangyang0918 commented on code in PR #155:
URL: https://github.com/apache/flink-kubernetes-operator/pull/155#discussion_r844570889
##########
docs/content/docs/operations/helm.md:
##########
@@ -109,3 +109,107 @@ spec:
skipCrds: true
...
```
+
+## Advanced customization techniques
+The Helm chart does not aim to provide configuration options for all the possible deployment scenarios of the Operator. There are use cases for injecting common tools and/or sidecars in most enterprise environments that simply cannot be covered by public Helm charts.
+
+Fortunately, [post rendering](https://helm.sh/docs/topics/advanced/#post-rendering) in Helm gives you the ability to manually manipulate manifests before they are installed on a Kubernetes cluster. This allows users to use tools like [kustomize](https://kustomize.io) to apply configuration changes without the need to fork public charts.
+
+The GitHub repository for the Operator contains a simple [example](https://github.com/apache/flink-kubernetes-operator/tree/main/examples/kustomize) on how to augment the Operator Deployment with a [fluent-bit](https://docs.fluentbit.io/manual) sidecar container and adjust container resources using `kustomize`.
+
+The example demonstrates that we can still use a `values.yaml` file to override the default Helm values for changing the log configuration, for example:
+
+```yaml
+
+operatorConfiguration:
+ ...
+ log4j2.properties: |+
+ rootLogger.appenderRef.file.ref = LogFile
+ appender.file.name = LogFile
+ appender.file.type = File
+ appender.file.append = false
+ appender.file.fileName = ${sys:log.file}
+ appender.file.layout.type = PatternLayout
+ appender.file.layout.pattern = %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %-60c %x - %m%n
+
+jvmArgs:
+ webhook: "-Dlog.file=/opt/flink/log/webhook.log -Xms256m -Xmx256m"
+ operator: "-Dlog.file=/opt/flink/log/operator.log -Xms2048m -Xmx2048m"
+```
+
+But we cannot ingest our fluent-bit sidecar for example unless we patch the deployment using `kustomize`
+
+```yaml
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+ name: not-important
+spec:
+ template:
+ spec:
+ containers:
+ - name: flink-kubernetes-operator
+ volumeMounts:
+ - name: flink-log
+ mountPath: /opt/flink/log
+ resources:
+ requests:
+ memory: "2.5Gi"
+ cpu: "1000m"
+ limits:
+ memory: "2.5Gi"
+ cpu: "2000m"
+ - name: flink-webhook
+ volumeMounts:
+ - name: flink-log
+ mountPath: /opt/flink/log
+ resources:
+ requests:
+ memory: "0.5Gi"
+ cpu: "200m"
+ limits:
+ memory: "0.5Gi"
+ cpu: "500m"
+ - name: fluentbit
+ image: fluent/fluent-bit:1.8.12
+ command: [ 'sh','-c','/fluent-bit/bin/fluent-bit -i tail -p path=/opt/flink/log/*.log -p multiline.parser=java -o stdout' ]
Review Comment:
It seems that the `sh` is not available in the image `fluent/fluent-bit:1.8.12`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-kubernetes-operator] wangyang0918 commented on pull request #155: [FLINK-26663] Pod augmentation for the operator
Posted by GitBox <gi...@apache.org>.
wangyang0918 commented on PR #155:
URL: https://github.com/apache/flink-kubernetes-operator/pull/155#issuecomment-1089863822
We need to clarify some behaviors.
* What's the priority of `--set` and `kustomize`?
* Could we use `kustomize` to decorate some existing container, e.g. adding resources for flink-kubernetes-operator or flink-webhook?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-kubernetes-operator] morhidi commented on a diff in pull request #155: [FLINK-26663] Pod augmentation for the operator
Posted by GitBox <gi...@apache.org>.
morhidi commented on code in PR #155:
URL: https://github.com/apache/flink-kubernetes-operator/pull/155#discussion_r844766533
##########
docs/content/docs/operations/helm.md:
##########
@@ -109,3 +109,107 @@ spec:
skipCrds: true
...
```
+
+## Advanced customization techniques
+The Helm chart does not aim to provide configuration options for all the possible deployment scenarios of the Operator. There are use cases for injecting common tools and/or sidecars in most enterprise environments that simply cannot be covered by public Helm charts.
+
+Fortunately, [post rendering](https://helm.sh/docs/topics/advanced/#post-rendering) in Helm gives you the ability to manually manipulate manifests before they are installed on a Kubernetes cluster. This allows users to use tools like [kustomize](https://kustomize.io) to apply configuration changes without the need to fork public charts.
+
+The GitHub repository for the Operator contains a simple [example](https://github.com/apache/flink-kubernetes-operator/tree/main/examples/kustomize) on how to augment the Operator Deployment with a [fluent-bit](https://docs.fluentbit.io/manual) sidecar container and adjust container resources using `kustomize`.
+
+The example demonstrates that we can still use a `values.yaml` file to override the default Helm values for changing the log configuration, for example:
+
+```yaml
+
+operatorConfiguration:
+ ...
+ log4j2.properties: |+
+ rootLogger.appenderRef.file.ref = LogFile
+ appender.file.name = LogFile
+ appender.file.type = File
+ appender.file.append = false
+ appender.file.fileName = ${sys:log.file}
+ appender.file.layout.type = PatternLayout
+ appender.file.layout.pattern = %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %-60c %x - %m%n
+
+jvmArgs:
+ webhook: "-Dlog.file=/opt/flink/log/webhook.log -Xms256m -Xmx256m"
+ operator: "-Dlog.file=/opt/flink/log/operator.log -Xms2048m -Xmx2048m"
+```
+
+But we cannot ingest our fluent-bit sidecar for example unless we patch the deployment using `kustomize`
+
+```yaml
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+ name: not-important
+spec:
+ template:
+ spec:
+ containers:
+ - name: flink-kubernetes-operator
+ volumeMounts:
+ - name: flink-log
+ mountPath: /opt/flink/log
+ resources:
+ requests:
+ memory: "2.5Gi"
+ cpu: "1000m"
+ limits:
+ memory: "2.5Gi"
+ cpu: "2000m"
+ - name: flink-webhook
+ volumeMounts:
+ - name: flink-log
+ mountPath: /opt/flink/log
+ resources:
+ requests:
+ memory: "0.5Gi"
+ cpu: "200m"
+ limits:
+ memory: "0.5Gi"
+ cpu: "500m"
+ - name: fluentbit
+ image: fluent/fluent-bit:1.8.12
+ command: [ 'sh','-c','/fluent-bit/bin/fluent-bit -i tail -p path=/opt/flink/log/*.log -p multiline.parser=java -o stdout' ]
Review Comment:
thanks @wangyang0918 bit weird, it worked on my arm64 arch laptop, changed it to the debug version in the example, anyway, to be able to attach a terminal to it. PTAL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-kubernetes-operator] wangyang0918 commented on a diff in pull request #155: [FLINK-26663] Pod augmentation for the operator
Posted by GitBox <gi...@apache.org>.
wangyang0918 commented on code in PR #155:
URL: https://github.com/apache/flink-kubernetes-operator/pull/155#discussion_r844937962
##########
docs/content/docs/operations/helm.md:
##########
@@ -109,3 +109,107 @@ spec:
skipCrds: true
...
```
+
+## Advanced customization techniques
+The Helm chart does not aim to provide configuration options for all the possible deployment scenarios of the Operator. There are use cases for injecting common tools and/or sidecars in most enterprise environments that simply cannot be covered by public Helm charts.
+
+Fortunately, [post rendering](https://helm.sh/docs/topics/advanced/#post-rendering) in Helm gives you the ability to manually manipulate manifests before they are installed on a Kubernetes cluster. This allows users to use tools like [kustomize](https://kustomize.io) to apply configuration changes without the need to fork public charts.
+
+The GitHub repository for the Operator contains a simple [example](https://github.com/apache/flink-kubernetes-operator/tree/main/examples/kustomize) on how to augment the Operator Deployment with a [fluent-bit](https://docs.fluentbit.io/manual) sidecar container and adjust container resources using `kustomize`.
+
+The example demonstrates that we can still use a `values.yaml` file to override the default Helm values for changing the log configuration, for example:
+
+```yaml
+
+operatorConfiguration:
+ ...
+ log4j2.properties: |+
+ rootLogger.appenderRef.file.ref = LogFile
+ appender.file.name = LogFile
+ appender.file.type = File
+ appender.file.append = false
+ appender.file.fileName = ${sys:log.file}
+ appender.file.layout.type = PatternLayout
+ appender.file.layout.pattern = %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %-60c %x - %m%n
+
+jvmArgs:
+ webhook: "-Dlog.file=/opt/flink/log/webhook.log -Xms256m -Xmx256m"
+ operator: "-Dlog.file=/opt/flink/log/operator.log -Xms2048m -Xmx2048m"
+```
+
+But we cannot ingest our fluent-bit sidecar for example unless we patch the deployment using `kustomize`
+
+```yaml
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+ name: not-important
+spec:
+ template:
+ spec:
+ containers:
+ - name: flink-kubernetes-operator
+ volumeMounts:
+ - name: flink-log
+ mountPath: /opt/flink/log
+ resources:
+ requests:
+ memory: "2.5Gi"
+ cpu: "1000m"
+ limits:
+ memory: "2.5Gi"
+ cpu: "2000m"
+ - name: flink-webhook
+ volumeMounts:
+ - name: flink-log
+ mountPath: /opt/flink/log
+ resources:
+ requests:
+ memory: "0.5Gi"
+ cpu: "200m"
+ limits:
+ memory: "0.5Gi"
+ cpu: "500m"
+ - name: fluentbit
+ image: fluent/fluent-bit:1.8.12
+ command: [ 'sh','-c','/fluent-bit/bin/fluent-bit -i tail -p path=/opt/flink/log/*.log -p multiline.parser=java -o stdout' ]
Review Comment:
Could you share me some light why the `kustomize/values.yaml` does not take effect in my environment? Both the logging and the jvm args does not work for me.
```
helm install flink-kubernetes-operator helm/flink-kubernetes-operator --post-renderer examples/kustomize/render
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-kubernetes-operator] morhidi commented on a diff in pull request #155: [FLINK-26663] Pod augmentation for the operator
Posted by GitBox <gi...@apache.org>.
morhidi commented on code in PR #155:
URL: https://github.com/apache/flink-kubernetes-operator/pull/155#discussion_r844966623
##########
docs/content/docs/operations/helm.md:
##########
@@ -109,3 +109,107 @@ spec:
skipCrds: true
...
```
+
+## Advanced customization techniques
+The Helm chart does not aim to provide configuration options for all the possible deployment scenarios of the Operator. There are use cases for injecting common tools and/or sidecars in most enterprise environments that simply cannot be covered by public Helm charts.
+
+Fortunately, [post rendering](https://helm.sh/docs/topics/advanced/#post-rendering) in Helm gives you the ability to manually manipulate manifests before they are installed on a Kubernetes cluster. This allows users to use tools like [kustomize](https://kustomize.io) to apply configuration changes without the need to fork public charts.
+
+The GitHub repository for the Operator contains a simple [example](https://github.com/apache/flink-kubernetes-operator/tree/main/examples/kustomize) on how to augment the Operator Deployment with a [fluent-bit](https://docs.fluentbit.io/manual) sidecar container and adjust container resources using `kustomize`.
+
+The example demonstrates that we can still use a `values.yaml` file to override the default Helm values for changing the log configuration, for example:
+
+```yaml
+
+operatorConfiguration:
+ ...
+ log4j2.properties: |+
+ rootLogger.appenderRef.file.ref = LogFile
+ appender.file.name = LogFile
+ appender.file.type = File
+ appender.file.append = false
+ appender.file.fileName = ${sys:log.file}
+ appender.file.layout.type = PatternLayout
+ appender.file.layout.pattern = %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %-60c %x - %m%n
+
+jvmArgs:
+ webhook: "-Dlog.file=/opt/flink/log/webhook.log -Xms256m -Xmx256m"
+ operator: "-Dlog.file=/opt/flink/log/operator.log -Xms2048m -Xmx2048m"
+```
+
+But we cannot ingest our fluent-bit sidecar for example unless we patch the deployment using `kustomize`
+
+```yaml
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+ name: not-important
+spec:
+ template:
+ spec:
+ containers:
+ - name: flink-kubernetes-operator
+ volumeMounts:
+ - name: flink-log
+ mountPath: /opt/flink/log
+ resources:
+ requests:
+ memory: "2.5Gi"
+ cpu: "1000m"
+ limits:
+ memory: "2.5Gi"
+ cpu: "2000m"
+ - name: flink-webhook
+ volumeMounts:
+ - name: flink-log
+ mountPath: /opt/flink/log
+ resources:
+ requests:
+ memory: "0.5Gi"
+ cpu: "200m"
+ limits:
+ memory: "0.5Gi"
+ cpu: "500m"
+ - name: fluentbit
+ image: fluent/fluent-bit:1.8.12
+ command: [ 'sh','-c','/fluent-bit/bin/fluent-bit -i tail -p path=/opt/flink/log/*.log -p multiline.parser=java -o stdout' ]
Review Comment:
bumped the fluent bit version to address the issue appears on latest macs
```
no matching manifest for linux/arm64/v8 in the manifest list entries.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org