You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@heron.apache.org by GitBox <gi...@apache.org> on 2022/07/21 18:47:45 UTC
[GitHub] [incubator-heron] surahman opened a new pull request, #3847: [3846] Refactoring of the K8s Shim
surahman opened a new pull request, #3847:
URL: https://github.com/apache/incubator-heron/pull/3847
### Refactoring of the K8s Shim
This issue is not related to a release and will be a code quality improvement. Closes #3846.
The [`V1Controller`](https://github.com/apache/incubator-heron/blob/master/heron/schedulers/src/java/org/apache/heron/scheduler/kubernetes/V1Controller.java) is a Shim to interface with the K8s cluster(s). It should only contain methods that support this functionality. All methods that communicate with the Cluster are retained in the `KubernetesShim` class. All methods that support the creation and configuration of the `Stateful Set`s are moved to the `StatefulSet` factory class. Data with cluster configurations and config maps are passed to the factory via the `Configs` container class.
#### __Kubernetes Shim:__
- `Constructor`
- `submit`
- `killTopology`
- `restart`
- `addContainers`
- `removeContainers`
- `patchStatefulSetReplicas`
- `getStatefulSet`
- `deleteService`
- `deleteStatefulSets`
- `createTopologyService`
- `getServiceAnnotations`
- `getPodMatchLabels`
- `getServiceLabels`
- `loadPodFromTemplate`
- `getPodTemplateLocation`
- `getConfigMap`
- `removePersistentVolumeClaims`
- `getPersistentVolumeClaimLabels`
- `getStatefulSetName`
- `createTopologySelectorLabels`
<br>
#### __StatefulSet (Factory):__
- `getExecutorCommand`
- `setShardIdEnvironmentVariableCommand`
- `createStatefulSet`
- `getPodAnnotations`
- `getPrometheusAnnotations`
- `getPodMatchLabels`
- `getPodLabels`
- `configurePodSpec`
- `configureTolerations`
- `getTolerations`
- `mountSecretsAsVolumes`
- `configureHeronContainer`
- `configureContainerResources`
- `createResourcesRequirement`
- `configureContainerEnvVars`
- `getExecutorEnvVars`
- `configureContainerPorts`
- `getExecutorPorts`
- `getDebuggingPorts`
- `mountVolumeIfPresent`
- `setSecretKeyRefs`
- `createPersistentVolumeClaims`
- `createVolumeAndMountsPersistentVolumeClaimCLI`
- `createVolumeAndMountsEmptyDirCLI`
- `createVolumeAndMountsHostPathCLI`
- `createVolumeAndMountsNFSCLI`
- `configurePodWithVolumesAndMountsFromCLI`
- `getPersistentVolumeClaimLabels`
- `getStatefulSetName`
<br>
<br>
#### __**Deployment Testing:**__
Deployment :heavy_check_mark:
Teardown :heavy_check_mark:
Cluster in good state :heavy_check_mark:
<details>
<summary>Command</summary>
```bash
~/bin/heron submit kubernetes ~/.heron/examples/heron-api-examples.jar \
org.apache.heron.examples.api.AckingTopology acking \
--verbose \
--deploy-deactivated \
--config-property heron.kubernetes.executor.pod.template=pod-templ-executor.pod-template-executor.yaml \
--config-property heron.kubernetes.manager.pod.template=pod-templ-manager.pod-template-manager.yaml \
--config-property heron.kubernetes.manager.limits.cpu=2 \
--config-property heron.kubernetes.manager.limits.memory=3 \
--config-property heron.kubernetes.manager.requests.cpu=1 \
--config-property heron.kubernetes.manager.requests.memory=2 \
--config-property heron.kubernetes.manager.volumes.persistentVolumeClaim.manager-dynamic-volume.claimName=OnDemand \
--config-property heron.kubernetes.manager.volumes.persistentVolumeClaim.manager-dynamic-volume.storageClassName=storage-class-name-manager \
--config-property heron.kubernetes.manager.volumes.persistentVolumeClaim.manager-dynamic-volume.accessModes=ReadWriteOnce,ReadOnlyMany \
--config-property heron.kubernetes.manager.volumes.persistentVolumeClaim.manager-dynamic-volume.sizeLimit=256Gi \
--config-property heron.kubernetes.manager.volumes.persistentVolumeClaim.manager-dynamic-volume.volumeMode=Block \
--config-property heron.kubernetes.manager.volumes.persistentVolumeClaim.manager-dynamic-volume.path=path/to/mount/dynamic/volume \
--config-property heron.kubernetes.manager.volumes.persistentVolumeClaim.manager-dynamic-volume.subPath=sub/path/to/mount/dynamic/volume \
\
--config-property heron.kubernetes.manager.volumes.persistentVolumeClaim.manager-static-volume.claimName=OnDemand \
--config-property heron.kubernetes.manager.volumes.persistentVolumeClaim.manager-static-volume.accessModes=ReadWriteOnce,ReadOnlyMany \
--config-property heron.kubernetes.manager.volumes.persistentVolumeClaim.manager-static-volume.sizeLimit=512Gi \
--config-property heron.kubernetes.manager.volumes.persistentVolumeClaim.manager-static-volume.volumeMode=Block \
--config-property heron.kubernetes.manager.volumes.persistentVolumeClaim.manager-static-volume.path=path/to/mount/static/volume \
--config-property heron.kubernetes.manager.volumes.persistentVolumeClaim.manager-static-volume.subPath=sub/path/to/mount/static/volume \
\
--config-property heron.kubernetes.manager.volumes.persistentVolumeClaim.manager-shared-volume.claimName=requested-claim-by-user \
--config-property heron.kubernetes.manager.volumes.persistentVolumeClaim.manager-shared-volume.path=path/to/mount/shared/volume \
--config-property heron.kubernetes.manager.volumes.persistentVolumeClaim.manager-shared-volume.subPath=sub/path/to/mount/shared/volume \
\
--config-property heron.kubernetes.manager.volumes.emptyDir.manager-empty-dir.medium="Memory" \
--config-property heron.kubernetes.manager.volumes.emptyDir.manager-empty-dir.sizeLimit="50Mi" \
--config-property heron.kubernetes.manager.volumes.emptyDir.manager-empty-dir.path="empty/dir/path" \
--config-property heron.kubernetes.manager.volumes.emptyDir.manager-empty-dir.subPath="empty/dir/sub/path" \
--config-property heron.kubernetes.manager.volumes.emptyDir.manager-empty-dir.readOnly="true" \
\
--config-property heron.kubernetes.manager.volumes.hostPath.manager-host-path.type="File" \
--config-property heron.kubernetes.manager.volumes.hostPath.manager-host-path.pathOnHost="/dev/null" \
--config-property heron.kubernetes.manager.volumes.hostPath.manager-host-path.path="host/path/path" \
--config-property heron.kubernetes.manager.volumes.hostPath.manager-host-path.subPath="host/path/sub/path" \
--config-property heron.kubernetes.manager.volumes.hostPath.manager-host-path.readOnly="true" \
\
--config-property heron.kubernetes.manager.volumes.nfs.manager-nfs.server="nfs-server.address" \
--config-property heron.kubernetes.manager.volumes.nfs.manager-nfs.readOnly="true" \
--config-property heron.kubernetes.manager.volumes.nfs.manager-nfs.pathOnNFS="/dev/null" \
--config-property heron.kubernetes.manager.volumes.nfs.manager-nfs.path="nfs/path" \
--config-property heron.kubernetes.manager.volumes.nfs.manager-nfs.subPath="nfs/sub/path" \
--config-property heron.kubernetes.manager.volumes.nfs.manager-nfs.readOnly="true" \
\
--config-property heron.kubernetes.executor.volumes.persistentVolumeClaim.executor-dynamic-volume.claimName=OnDemand \
--config-property heron.kubernetes.executor.volumes.persistentVolumeClaim.executor-dynamic-volume.storageClassName=storage-class-name-executor \
--config-property heron.kubernetes.executor.volumes.persistentVolumeClaim.executor-dynamic-volume.accessModes=ReadWriteOnce,ReadOnlyMany \
--config-property heron.kubernetes.executor.volumes.persistentVolumeClaim.executor-dynamic-volume.sizeLimit=256Gi \
--config-property heron.kubernetes.executor.volumes.persistentVolumeClaim.executor-dynamic-volume.volumeMode=Block \
--config-property heron.kubernetes.executor.volumes.persistentVolumeClaim.executor-dynamic-volume.path=path/to/mount/dynamic/volume \
--config-property heron.kubernetes.executor.volumes.persistentVolumeClaim.executor-dynamic-volume.subPath=sub/path/to/mount/dynamic/volume \
\
--config-property heron.kubernetes.executor.volumes.persistentVolumeClaim.executor-static-volume.claimName=OnDemand \
--config-property heron.kubernetes.executor.volumes.persistentVolumeClaim.executor-static-volume.accessModes=ReadWriteOnce,ReadOnlyMany \
--config-property heron.kubernetes.executor.volumes.persistentVolumeClaim.executor-static-volume.sizeLimit=512Gi \
--config-property heron.kubernetes.executor.volumes.persistentVolumeClaim.executor-static-volume.volumeMode=Block \
--config-property heron.kubernetes.executor.volumes.persistentVolumeClaim.executor-static-volume.path=path/to/mount/static/volume \
--config-property heron.kubernetes.executor.volumes.persistentVolumeClaim.executor-static-volume.subPath=sub/path/to/mount/static/volume \
\
--config-property heron.kubernetes.executor.volumes.persistentVolumeClaim.executor-shared-volume.claimName=requested-claim-by-user \
--config-property heron.kubernetes.executor.volumes.persistentVolumeClaim.executor-shared-volume.path=path/to/mount/shared/volume \
--config-property heron.kubernetes.executor.volumes.persistentVolumeClaim.executor-shared-volume.subPath=sub/path/to/mount/shared/volume
```
</details>
<details>
<summary>Stateful Sets</summary>
__*Executor:*__
```yaml
apiVersion: v1
items:
- apiVersion: apps/v1
kind: StatefulSet
metadata:
creationTimestamp: "2022-07-21T17:53:10Z"
generation: 1
labels:
app: heron
topology: acking
name: acking-executor
namespace: default
resourceVersion: "859"
uid: a946bf3d-88df-4320-9752-40e6d0327959
spec:
podManagementPolicy: Parallel
replicas: 2
revisionHistoryLimit: 10
selector:
matchLabels:
app: heron
topology: acking
serviceName: acking
template:
metadata:
annotations:
prometheus.io/port: "8080"
prometheus.io/scrape: "true"
creationTimestamp: null
labels:
app: heron
topology: acking
spec:
containers:
- command:
- sh
- -c
- './heron-core/bin/heron-downloader-config kubernetes && ./heron-core/bin/heron-downloader
distributedlog://zookeeper:2181/heronbkdl/acking-saad-tag-0.tar.gz . &&
SHARD_ID=$((${POD_NAME##*-} + 1)) && echo shardId=${SHARD_ID} && ./heron-core/bin/heron-executor
--topology-name=acking --topology-id=acking3255f000-6d02-476a-840d-9a3ca4338885
--topology-defn-file=acking.defn --state-manager-connection=zookeeper:2181
--state-manager-root=/heron --state-manager-config-file=./heron-conf/statemgr.yaml
--tmanager-binary=./heron-core/bin/heron-tmanager --stmgr-binary=./heron-core/bin/heron-stmgr
--metrics-manager-classpath=./heron-core/lib/metricsmgr/* --instance-jvm-opts="LVhYOitIZWFwRHVtcE9uT3V0T2ZNZW1vcnlFcnJvcg(61)(61)"
--classpath=heron-api-examples.jar --heron-internals-config-file=./heron-conf/heron_internals.yaml
--override-config-file=./heron-conf/override.yaml --component-ram-map=exclaim1:1073741824,word:1073741824
--component-jvm-opts="" --pkg-type=jar --topology-binary-file=heron-api-examples.jar
--heron-java-home=$JAVA_HOME --heron-shell-binary=./heron-core/bin/heron-shell
--cluster=kubernetes --role=saad --environment=default --instance-classpath=./heron-core/lib/instance/*
--metrics-sinks-config-file=./heron-conf/metrics_sinks.yaml --scheduler-classpath=./heron-core/lib/scheduler/*:./heron-core/lib/packing/*:./heron-core/lib/statemgr/*
--python-instance-binary=./heron-core/bin/heron-python-instance --cpp-instance-binary=./heron-core/bin/heron-cpp-instance
--metricscache-manager-classpath=./heron-core/lib/metricscachemgr/* --metricscache-manager-mode=disabled
--is-stateful=false --checkpoint-manager-classpath=./heron-core/lib/ckptmgr/*:./heron-core/lib/statefulstorage/*:
--stateful-config-file=./heron-conf/stateful.yaml --checkpoint-manager-ram=1073741824
--health-manager-mode=disabled --health-manager-classpath=./heron-core/lib/healthmgr/*
--shard=$SHARD_ID --server-port=6001 --tmanager-controller-port=6002 --tmanager-stats-port=6003
--shell-port=6004 --metrics-manager-port=6005 --scheduler-port=6006 --metricscache-manager-server-port=6007
--metricscache-manager-stats-port=6008 --checkpoint-manager-port=6009'
env:
- name: HOST
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: status.podIP
- name: POD_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.name
- name: var_one
value: variable one
- name: var_three
value: variable three
- name: var_two
value: variable two
image: apache/heron:testbuild
imagePullPolicy: IfNotPresent
name: executor
ports:
- containerPort: 5555
name: tcp-port-kept
protocol: TCP
- containerPort: 5556
name: udp-port-kept
protocol: UDP
- containerPort: 6001
name: server
protocol: TCP
- containerPort: 6002
name: tmanager-ctl
protocol: TCP
- containerPort: 6003
name: tmanager-stats
protocol: TCP
- containerPort: 6004
name: shell-port
protocol: TCP
- containerPort: 6005
name: metrics-mgr
protocol: TCP
- containerPort: 6006
name: scheduler
protocol: TCP
- containerPort: 6007
name: metrics-cache-m
protocol: TCP
- containerPort: 6008
name: metrics-cache-s
protocol: TCP
- containerPort: 6009
name: ckptmgr
protocol: TCP
resources:
limits:
cpu: "3"
memory: 4Gi
requests:
cpu: "3"
memory: 4Gi
securityContext:
allowPrivilegeEscalation: false
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: path/to/mount/dynamic/volume
name: executor-dynamic-volume
subPath: sub/path/to/mount/dynamic/volume
- mountPath: path/to/mount/shared/volume
name: executor-shared-volume
subPath: sub/path/to/mount/shared/volume
- mountPath: path/to/mount/static/volume
name: executor-static-volume
subPath: sub/path/to/mount/static/volume
- mountPath: /shared_volume
name: shared-volume
- image: alpine
imagePullPolicy: Always
name: sidecar-container
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /shared_volume
name: shared-volume
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
terminationGracePeriodSeconds: 0
tolerations:
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
tolerationSeconds: 10
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
tolerationSeconds: 10
volumes:
- name: executor-shared-volume
persistentVolumeClaim:
claimName: requested-claim-by-user
- emptyDir: {}
name: shared-volume
updateStrategy:
rollingUpdate:
partition: 0
type: RollingUpdate
volumeClaimTemplates:
- apiVersion: v1
kind: PersistentVolumeClaim
metadata:
creationTimestamp: null
labels:
onDemand: "true"
topology: acking
name: executor-dynamic-volume
spec:
accessModes:
- ReadWriteOnce
- ReadOnlyMany
resources:
requests:
storage: 256Gi
storageClassName: storage-class-name-executor
volumeMode: Block
status:
phase: Pending
- apiVersion: v1
kind: PersistentVolumeClaim
metadata:
creationTimestamp: null
labels:
onDemand: "true"
topology: acking
name: executor-static-volume
spec:
accessModes:
- ReadWriteOnce
- ReadOnlyMany
resources:
requests:
storage: 512Gi
storageClassName: ""
volumeMode: Block
status:
phase: Pending
status:
collisionCount: 0
currentReplicas: 2
currentRevision: acking-executor-69684dd444
observedGeneration: 1
replicas: 2
updateRevision: acking-executor-69684dd444
updatedReplicas: 2
```
__*Manager:*__
```yaml
- apiVersion: apps/v1
kind: StatefulSet
metadata:
creationTimestamp: "2022-07-21T17:53:10Z"
generation: 1
labels:
app: heron
topology: acking
name: acking-manager
namespace: default
resourceVersion: "867"
uid: 83590047-23dc-44d4-9244-23176e62f8bd
spec:
podManagementPolicy: Parallel
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app: heron
topology: acking
serviceName: acking
template:
metadata:
annotations:
prometheus.io/port: "8080"
prometheus.io/scrape: "true"
creationTimestamp: null
labels:
app: heron
topology: acking
spec:
containers:
- command:
- sh
- -c
- './heron-core/bin/heron-downloader-config kubernetes && ./heron-core/bin/heron-downloader
distributedlog://zookeeper:2181/heronbkdl/acking-saad-tag-0.tar.gz . &&
SHARD_ID=${POD_NAME##*-} && echo shardId=${SHARD_ID} && ./heron-core/bin/heron-executor
--topology-name=acking --topology-id=acking3255f000-6d02-476a-840d-9a3ca4338885
--topology-defn-file=acking.defn --state-manager-connection=zookeeper:2181
--state-manager-root=/heron --state-manager-config-file=./heron-conf/statemgr.yaml
--tmanager-binary=./heron-core/bin/heron-tmanager --stmgr-binary=./heron-core/bin/heron-stmgr
--metrics-manager-classpath=./heron-core/lib/metricsmgr/* --instance-jvm-opts="LVhYOitIZWFwRHVtcE9uT3V0T2ZNZW1vcnlFcnJvcg(61)(61)"
--classpath=heron-api-examples.jar --heron-internals-config-file=./heron-conf/heron_internals.yaml
--override-config-file=./heron-conf/override.yaml --component-ram-map=exclaim1:1073741824,word:1073741824
--component-jvm-opts="" --pkg-type=jar --topology-binary-file=heron-api-examples.jar
--heron-java-home=$JAVA_HOME --heron-shell-binary=./heron-core/bin/heron-shell
--cluster=kubernetes --role=saad --environment=default --instance-classpath=./heron-core/lib/instance/*
--metrics-sinks-config-file=./heron-conf/metrics_sinks.yaml --scheduler-classpath=./heron-core/lib/scheduler/*:./heron-core/lib/packing/*:./heron-core/lib/statemgr/*
--python-instance-binary=./heron-core/bin/heron-python-instance --cpp-instance-binary=./heron-core/bin/heron-cpp-instance
--metricscache-manager-classpath=./heron-core/lib/metricscachemgr/* --metricscache-manager-mode=disabled
--is-stateful=false --checkpoint-manager-classpath=./heron-core/lib/ckptmgr/*:./heron-core/lib/statefulstorage/*:
--stateful-config-file=./heron-conf/stateful.yaml --checkpoint-manager-ram=1073741824
--health-manager-mode=disabled --health-manager-classpath=./heron-core/lib/healthmgr/*
--shard=$SHARD_ID --server-port=6001 --tmanager-controller-port=6002 --tmanager-stats-port=6003
--shell-port=6004 --metrics-manager-port=6005 --scheduler-port=6006 --metricscache-manager-server-port=6007
--metricscache-manager-stats-port=6008 --checkpoint-manager-port=6009'
env:
- name: HOST
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: status.podIP
- name: POD_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.name
- name: var_one_manager
value: variable one on manager
- name: var_three_manager
value: variable three on manager
- name: var_two_manager
value: variable two on manager
image: apache/heron:testbuild
imagePullPolicy: IfNotPresent
name: manager
ports:
- containerPort: 6001
name: server
protocol: TCP
- containerPort: 6002
name: tmanager-ctl
protocol: TCP
- containerPort: 6003
name: tmanager-stats
protocol: TCP
- containerPort: 6004
name: shell-port
protocol: TCP
- containerPort: 6005
name: metrics-mgr
protocol: TCP
- containerPort: 6006
name: scheduler
protocol: TCP
- containerPort: 6007
name: metrics-cache-m
protocol: TCP
- containerPort: 6008
name: metrics-cache-s
protocol: TCP
- containerPort: 6009
name: ckptmgr
protocol: TCP
- containerPort: 7775
name: tcp-port-kept
protocol: TCP
- containerPort: 7776
name: udp-port-kept
protocol: UDP
resources:
limits:
cpu: "2"
memory: "3"
requests:
cpu: "1"
memory: "2"
securityContext:
allowPrivilegeEscalation: false
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: path/to/mount/dynamic/volume
name: manager-dynamic-volume
subPath: sub/path/to/mount/dynamic/volume
- mountPath: empty/dir/path
name: manager-empty-dir
readOnly: true
subPath: empty/dir/sub/path
- mountPath: host/path/path
name: manager-host-path
readOnly: true
subPath: host/path/sub/path
- mountPath: nfs/path
name: manager-nfs
readOnly: true
subPath: nfs/sub/path
- mountPath: path/to/mount/shared/volume
name: manager-shared-volume
subPath: sub/path/to/mount/shared/volume
- mountPath: path/to/mount/static/volume
name: manager-static-volume
subPath: sub/path/to/mount/static/volume
- mountPath: /shared_volume/manager
name: shared-volume-manager
- image: alpine
imagePullPolicy: Always
name: manager-sidecar-container
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /shared_volume/manager
name: shared-volume-manager
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
terminationGracePeriodSeconds: 0
tolerations:
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
tolerationSeconds: 10
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
tolerationSeconds: 10
volumes:
- emptyDir:
medium: Memory
sizeLimit: 50Mi
name: manager-empty-dir
- hostPath:
path: /dev/null
type: File
name: manager-host-path
- name: manager-nfs
nfs:
path: /dev/null
readOnly: true
server: nfs-server.address
- name: manager-shared-volume
persistentVolumeClaim:
claimName: requested-claim-by-user
- emptyDir: {}
name: shared-volume-manager
updateStrategy:
rollingUpdate:
partition: 0
type: RollingUpdate
volumeClaimTemplates:
- apiVersion: v1
kind: PersistentVolumeClaim
metadata:
creationTimestamp: null
labels:
onDemand: "true"
topology: acking
name: manager-static-volume
spec:
accessModes:
- ReadWriteOnce
- ReadOnlyMany
resources:
requests:
storage: 512Gi
storageClassName: ""
volumeMode: Block
status:
phase: Pending
- apiVersion: v1
kind: PersistentVolumeClaim
metadata:
creationTimestamp: null
labels:
onDemand: "true"
topology: acking
name: manager-dynamic-volume
spec:
accessModes:
- ReadWriteOnce
- ReadOnlyMany
resources:
requests:
storage: 256Gi
storageClassName: storage-class-name-manager
volumeMode: Block
status:
phase: Pending
status:
collisionCount: 0
currentReplicas: 1
currentRevision: acking-manager-86689946f9
observedGeneration: 1
replicas: 1
updateRevision: acking-manager-86689946f9
updatedReplicas: 1
- apiVersion: apps/v1
kind: StatefulSet
metadata:
creationTimestamp: "2022-07-21T17:50:57Z"
generation: 1
labels:
app: heron
component: zookeeper
name: zk
namespace: default
resourceVersion: "639"
uid: 4acf3017-252e-457d-966c-a1d85268ed5f
spec:
podManagementPolicy: OrderedReady
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app: heron
component: zookeeper
serviceName: zookeeper
template:
metadata:
creationTimestamp: null
labels:
app: heron
component: zookeeper
spec:
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values:
- zookeeper
topologyKey: kubernetes.io/hostname
weight: 1
containers:
- args:
- |
/opt/zookeeper/scripts/start-zookeeper.sh
command:
- sh
- -c
env:
- name: ZOOKEEPER_SERVERS
value: zk-0
image: apache/heron:testbuild
imagePullPolicy: IfNotPresent
livenessProbe:
exec:
command:
- /opt/zookeeper/scripts/zookeeper-ruok.sh
failureThreshold: 3
initialDelaySeconds: 15
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
name: zookeeper
ports:
- containerPort: 2181
name: client
protocol: TCP
- containerPort: 2888
name: server
protocol: TCP
- containerPort: 3888
name: leader-election
protocol: TCP
readinessProbe:
exec:
command:
- /opt/zookeeper/scripts/zookeeper-ruok.sh
failureThreshold: 3
initialDelaySeconds: 5
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /heron/data
name: datadir
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
terminationGracePeriodSeconds: 30
volumes:
- emptyDir: {}
name: datadir
updateStrategy:
rollingUpdate:
partition: 0
type: RollingUpdate
status:
availableReplicas: 1
collisionCount: 0
currentReplicas: 1
currentRevision: zk-c688744d8
observedGeneration: 1
readyReplicas: 1
replicas: 1
updateRevision: zk-c688744d8
updatedReplicas: 1
kind: List
metadata:
resourceVersion: ""
selfLink: ""
```
</details>
<details>
<summary>Persistent Volume Claims</summary>
```yaml
apiVersion: v1
items:
- apiVersion: v1
kind: PersistentVolumeClaim
metadata:
creationTimestamp: "2022-07-21T17:53:10Z"
finalizers:
- kubernetes.io/pvc-protection
labels:
app: heron
onDemand: "true"
topology: acking
name: executor-dynamic-volume-acking-executor-0
namespace: default
resourceVersion: "838"
uid: 2fe7e176-5821-46fb-bc09-fd2e76793690
spec:
accessModes:
- ReadWriteOnce
- ReadOnlyMany
resources:
requests:
storage: 256Gi
storageClassName: storage-class-name-executor
volumeMode: Block
status:
phase: Pending
- apiVersion: v1
kind: PersistentVolumeClaim
metadata:
creationTimestamp: "2022-07-21T17:53:10Z"
finalizers:
- kubernetes.io/pvc-protection
labels:
app: heron
onDemand: "true"
topology: acking
name: executor-dynamic-volume-acking-executor-1
namespace: default
resourceVersion: "845"
uid: cb8b7907-7fba-4603-9864-3da8f9e7c19c
spec:
accessModes:
- ReadWriteOnce
- ReadOnlyMany
resources:
requests:
storage: 256Gi
storageClassName: storage-class-name-executor
volumeMode: Block
status:
phase: Pending
- apiVersion: v1
kind: PersistentVolumeClaim
metadata:
creationTimestamp: "2022-07-21T17:53:10Z"
finalizers:
- kubernetes.io/pvc-protection
labels:
app: heron
onDemand: "true"
topology: acking
name: executor-static-volume-acking-executor-0
namespace: default
resourceVersion: "839"
uid: 8dd24cec-bd3c-4a00-b5e3-ec6087535b81
spec:
accessModes:
- ReadWriteOnce
- ReadOnlyMany
resources:
requests:
storage: 512Gi
storageClassName: ""
volumeMode: Block
status:
phase: Pending
- apiVersion: v1
kind: PersistentVolumeClaim
metadata:
creationTimestamp: "2022-07-21T17:53:10Z"
finalizers:
- kubernetes.io/pvc-protection
labels:
app: heron
onDemand: "true"
topology: acking
name: executor-static-volume-acking-executor-1
namespace: default
resourceVersion: "849"
uid: 9e67b08f-df1c-454c-8eda-fef5a46d7fd5
spec:
accessModes:
- ReadWriteOnce
- ReadOnlyMany
resources:
requests:
storage: 512Gi
storageClassName: ""
volumeMode: Block
status:
phase: Pending
- apiVersion: v1
kind: PersistentVolumeClaim
metadata:
creationTimestamp: "2022-07-21T17:53:10Z"
finalizers:
- kubernetes.io/pvc-protection
labels:
app: heron
onDemand: "true"
topology: acking
name: manager-dynamic-volume-acking-manager-0
namespace: default
resourceVersion: "856"
uid: 2457796c-2274-4e3a-b786-324895b8e979
spec:
accessModes:
- ReadWriteOnce
- ReadOnlyMany
resources:
requests:
storage: 256Gi
storageClassName: storage-class-name-manager
volumeMode: Block
status:
phase: Pending
- apiVersion: v1
kind: PersistentVolumeClaim
metadata:
creationTimestamp: "2022-07-21T17:53:10Z"
finalizers:
- kubernetes.io/pvc-protection
labels:
app: heron
onDemand: "true"
topology: acking
name: manager-static-volume-acking-manager-0
namespace: default
resourceVersion: "850"
uid: 7174e5b4-1079-48ee-90fb-0ddc8f08d6d0
spec:
accessModes:
- ReadWriteOnce
- ReadOnlyMany
resources:
requests:
storage: 512Gi
storageClassName: ""
volumeMode: Block
status:
phase: Pending
kind: List
metadata:
resourceVersion: ""
selfLink: ""
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@heron.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-heron] surahman commented on pull request #3847: [3846] Refactoring of the K8s Shim
Posted by GitBox <gi...@apache.org>.
surahman commented on PR #3847:
URL: https://github.com/apache/incubator-heron/pull/3847#issuecomment-1193346781
Thank you @joshfischer1108 and @nicknezis
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@heron.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-heron] nicknezis merged pull request #3847: [3846] Refactoring of the K8s Shim
Posted by GitBox <gi...@apache.org>.
nicknezis merged PR #3847:
URL: https://github.com/apache/incubator-heron/pull/3847
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@heron.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org