You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@heron.apache.org by GitBox <gi...@apache.org> on 2022/05/03 16:42:52 UTC
[GitHub] [incubator-heron] surahman commented on pull request #3825: [3821] Remove deprecated Host Path, NFS, and EBS support for Kubernetes
surahman commented on PR #3825:
URL: https://github.com/apache/incubator-heron/pull/3825#issuecomment-1116312639
Good deployment test. `Submit` and `Kill` are successful and all created resources are added and removed.
<details>
<summary>Command</summary>
```bash
~/bin/heron submit kubernetes ~/.heron/examples/heron-api-examples.jar \
org.apache.heron.examples.api.AckingTopology acking \
--verbose \
--deploy-deactivated \
--config-property heron.kubernetes.executor.pod.template=pod-templ-executor.pod-template-executor.yaml \
--config-property heron.kubernetes.manager.pod.template=pod-templ-manager.pod-template-manager.yaml \
--config-property heron.kubernetes.manager.limits.cpu=2 \
--config-property heron.kubernetes.manager.limits.memory=3 \
--config-property heron.kubernetes.manager.requests.cpu=1 \
--config-property heron.kubernetes.manager.requests.memory=2 \
--config-property heron.kubernetes.manager.volumes.persistentVolumeClaim.manager-dynamic-volume.claimName=OnDemand \
--config-property heron.kubernetes.manager.volumes.persistentVolumeClaim.manager-dynamic-volume.storageClassName=storage-class-name-manager \
--config-property heron.kubernetes.manager.volumes.persistentVolumeClaim.manager-dynamic-volume.accessModes=ReadWriteOnce,ReadOnlyMany \
--config-property heron.kubernetes.manager.volumes.persistentVolumeClaim.manager-dynamic-volume.sizeLimit=256Gi \
--config-property heron.kubernetes.manager.volumes.persistentVolumeClaim.manager-dynamic-volume.volumeMode=Block \
--config-property heron.kubernetes.manager.volumes.persistentVolumeClaim.manager-dynamic-volume.path=path/to/mount/dynamic/volume \
--config-property heron.kubernetes.manager.volumes.persistentVolumeClaim.manager-dynamic-volume.subPath=sub/path/to/mount/dynamic/volume \
\
--config-property heron.kubernetes.manager.volumes.persistentVolumeClaim.manager-static-volume.claimName=OnDemand \
--config-property heron.kubernetes.manager.volumes.persistentVolumeClaim.manager-static-volume.accessModes=ReadWriteOnce,ReadOnlyMany \
--config-property heron.kubernetes.manager.volumes.persistentVolumeClaim.manager-static-volume.sizeLimit=512Gi \
--config-property heron.kubernetes.manager.volumes.persistentVolumeClaim.manager-static-volume.volumeMode=Block \
--config-property heron.kubernetes.manager.volumes.persistentVolumeClaim.manager-static-volume.path=path/to/mount/static/volume \
--config-property heron.kubernetes.manager.volumes.persistentVolumeClaim.manager-static-volume.subPath=sub/path/to/mount/static/volume \
\
--config-property heron.kubernetes.manager.volumes.persistentVolumeClaim.manager-shared-volume.claimName=requested-claim-by-user \
--config-property heron.kubernetes.manager.volumes.persistentVolumeClaim.manager-shared-volume.path=path/to/mount/shared/volume \
--config-property heron.kubernetes.manager.volumes.persistentVolumeClaim.manager-shared-volume.subPath=sub/path/to/mount/shared/volume \
\
--config-property heron.kubernetes.manager.volumes.emptyDir.manager-empty-dir.medium="Memory" \
--config-property heron.kubernetes.manager.volumes.emptyDir.manager-empty-dir.sizeLimit="50Mi" \
--config-property heron.kubernetes.manager.volumes.emptyDir.manager-empty-dir.path="empty/dir/path" \
--config-property heron.kubernetes.manager.volumes.emptyDir.manager-empty-dir.subPath="empty/dir/sub/path" \
--config-property heron.kubernetes.manager.volumes.emptyDir.manager-empty-dir.readOnly="true" \
\
--config-property heron.kubernetes.manager.volumes.hostPath.manager-host-path.type="File" \
--config-property heron.kubernetes.manager.volumes.hostPath.manager-host-path.pathOnHost="/dev/null" \
--config-property heron.kubernetes.manager.volumes.hostPath.manager-host-path.path="host/path/path" \
--config-property heron.kubernetes.manager.volumes.hostPath.manager-host-path.subPath="host/path/sub/path" \
--config-property heron.kubernetes.manager.volumes.hostPath.manager-host-path.readOnly="true" \
\
--config-property heron.kubernetes.manager.volumes.nfs.manager-nfs.server="nfs-server.address" \
--config-property heron.kubernetes.manager.volumes.nfs.manager-nfs.readOnly="true" \
--config-property heron.kubernetes.manager.volumes.nfs.manager-nfs.pathOnNFS="/dev/null" \
--config-property heron.kubernetes.manager.volumes.nfs.manager-nfs.path="nfs/path" \
--config-property heron.kubernetes.manager.volumes.nfs.manager-nfs.subPath="nfs/sub/path" \
--config-property heron.kubernetes.manager.volumes.nfs.manager-nfs.readOnly="true" \
\
--config-property heron.kubernetes.executor.volumes.persistentVolumeClaim.executor-dynamic-volume.claimName=OnDemand \
--config-property heron.kubernetes.executor.volumes.persistentVolumeClaim.executor-dynamic-volume.storageClassName=storage-class-name-executor \
--config-property heron.kubernetes.executor.volumes.persistentVolumeClaim.executor-dynamic-volume.accessModes=ReadWriteOnce,ReadOnlyMany \
--config-property heron.kubernetes.executor.volumes.persistentVolumeClaim.executor-dynamic-volume.sizeLimit=256Gi \
--config-property heron.kubernetes.executor.volumes.persistentVolumeClaim.executor-dynamic-volume.volumeMode=Block \
--config-property heron.kubernetes.executor.volumes.persistentVolumeClaim.executor-dynamic-volume.path=path/to/mount/dynamic/volume \
--config-property heron.kubernetes.executor.volumes.persistentVolumeClaim.executor-dynamic-volume.subPath=sub/path/to/mount/dynamic/volume \
\
--config-property heron.kubernetes.executor.volumes.persistentVolumeClaim.executor-static-volume.claimName=OnDemand \
--config-property heron.kubernetes.executor.volumes.persistentVolumeClaim.executor-static-volume.accessModes=ReadWriteOnce,ReadOnlyMany \
--config-property heron.kubernetes.executor.volumes.persistentVolumeClaim.executor-static-volume.sizeLimit=512Gi \
--config-property heron.kubernetes.executor.volumes.persistentVolumeClaim.executor-static-volume.volumeMode=Block \
--config-property heron.kubernetes.executor.volumes.persistentVolumeClaim.executor-static-volume.path=path/to/mount/static/volume \
--config-property heron.kubernetes.executor.volumes.persistentVolumeClaim.executor-static-volume.subPath=sub/path/to/mount/static/volume \
\
--config-property heron.kubernetes.executor.volumes.persistentVolumeClaim.executor-shared-volume.claimName=requested-claim-by-user \
--config-property heron.kubernetes.executor.volumes.persistentVolumeClaim.executor-shared-volume.path=path/to/mount/shared/volume \
--config-property heron.kubernetes.executor.volumes.persistentVolumeClaim.executor-shared-volume.subPath=sub/path/to/mount/shared/volume
```
</details>
<details>
<summary>Stateful Sets</summary>
#### Manager
```bash
Name: acking-manager
Namespace: default
CreationTimestamp: Tue, 03 May 2022 11:33:31 -0400
Selector: app=heron,topology=acking
Labels: app=heron
topology=acking
Annotations: <none>
Replicas: 1 desired | 1 total
Update Strategy: RollingUpdate
Partition: 0
Pods Status: 0 Running / 1 Waiting / 0 Succeeded / 0 Failed
Pod Template:
Labels: app=heron
topology=acking
Annotations: prometheus.io/port: 8080
prometheus.io/scrape: true
Containers:
manager:
Image: apache/heron:testbuild
Ports: 6001/TCP, 6002/TCP, 6003/TCP, 6004/TCP, 6005/TCP, 6006/TCP, 6007/TCP, 6008/TCP, 6009/TCP, 7775/TCP, 7776/UDP
Host Ports: 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/UDP
Command:
sh
-c
./heron-core/bin/heron-downloader-config kubernetes && ./heron-core/bin/heron-downloader distributedlog://zookeeper:2181/heronbkdl/acking-saad-tag-0--3944535742942496843.tar.gz . && SHARD_ID=${POD_NAME##*-} && echo shardId=${SHARD_ID} && ./heron-core/bin/heron-executor --topology-name=acking --topology-id=acking3ae6b6a0-bbb8-45a1-8015-37fd50fec7c6 --topology-defn-file=acking.defn --state-manager-connection=zookeeper:2181 --state-manager-root=/heron --state-manager-config-file=./heron-conf/statemgr.yaml --tmanager-binary=./heron-core/bin/heron-tmanager --stmgr-binary=./heron-core/bin/heron-stmgr --metrics-manager-classpath=./heron-core/lib/metricsmgr/* --instance-jvm-opts="LVhYOitIZWFwRHVtcE9uT3V0T2ZNZW1vcnlFcnJvcg(61)(61)" --classpath=heron-api-examples.jar --heron-internals-config-file=./heron-conf/heron_internals.yaml --override-config-file=./heron-conf/override.yaml --component-ram-map=exclaim1:1073741824,word:1073741824 --component-jvm-opts="" --pkg-type=jar --topology-b
inary-file=heron-api-examples.jar --heron-java-home=$JAVA_HOME --heron-shell-binary=./heron-core/bin/heron-shell --cluster=kubernetes --role=saad --environment=default --instance-classpath=./heron-core/lib/instance/* --metrics-sinks-config-file=./heron-conf/metrics_sinks.yaml --scheduler-classpath=./heron-core/lib/scheduler/*:./heron-core/lib/packing/*:./heron-core/lib/statemgr/* --python-instance-binary=./heron-core/bin/heron-python-instance --cpp-instance-binary=./heron-core/bin/heron-cpp-instance --metricscache-manager-classpath=./heron-core/lib/metricscachemgr/* --metricscache-manager-mode=disabled --is-stateful=false --checkpoint-manager-classpath=./heron-core/lib/ckptmgr/*:./heron-core/lib/statefulstorage/*: --stateful-config-file=./heron-conf/stateful.yaml --checkpoint-manager-ram=1073741824 --health-manager-mode=disabled --health-manager-classpath=./heron-core/lib/healthmgr/* --shard=$SHARD_ID --server-port=6001 --tmanager-controller-port=6002 --tmanager-stats-port=6003 --sh
ell-port=6004 --metrics-manager-port=6005 --scheduler-port=6006 --metricscache-manager-server-port=6007 --metricscache-manager-stats-port=6008 --checkpoint-manager-port=6009
Limits:
cpu: 2
memory: 3
Requests:
cpu: 1
memory: 2
Environment:
HOST: (v1:status.podIP)
POD_NAME: (v1:metadata.name)
var_one_manager: variable one on manager
var_three_manager: variable three on manager
var_two_manager: variable two on manager
Mounts:
/shared_volume/manager from shared-volume-manager (rw)
empty/dir/path from manager-empty-dir (ro,path="empty/dir/sub/path")
host/path/path from manager-host-path (ro,path="host/path/sub/path")
nfs/path from manager-nfs (ro,path="nfs/sub/path")
path/to/mount/dynamic/volume from manager-dynamic-volume (rw,path="sub/path/to/mount/dynamic/volume")
path/to/mount/shared/volume from manager-shared-volume (rw,path="sub/path/to/mount/shared/volume")
path/to/mount/static/volume from manager-static-volume (rw,path="sub/path/to/mount/static/volume")
manager-sidecar-container:
Image: alpine
Port: <none>
Host Port: <none>
Environment: <none>
Mounts:
/shared_volume/manager from shared-volume-manager (rw)
Volumes:
manager-empty-dir:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium: Memory
SizeLimit: 50Mi
manager-host-path:
Type: HostPath (bare host directory volume)
Path: /dev/null
HostPathType: File
manager-nfs:
Type: NFS (an NFS mount that lasts the lifetime of a pod)
Server: nfs-server.address
Path: /dev/null
ReadOnly: true
manager-shared-volume:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: requested-claim-by-user
ReadOnly: false
shared-volume-manager:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
Volume Claims:
Name: manager-static-volume
StorageClass:
Labels: onDemand=true
topology=acking
Annotations: <none>
Capacity: 512Gi
Access Modes: [ReadWriteOnce ReadOnlyMany]
Name: manager-dynamic-volume
StorageClass: storage-class-name-manager
Labels: onDemand=true
topology=acking
Annotations: <none>
Capacity: 256Gi
Access Modes: [ReadWriteOnce ReadOnlyMany]
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal SuccessfulCreate 6m38s statefulset-controller create Claim manager-static-volume-acking-manager-0 Pod acking-manager-0 in StatefulSet acking-manager success
Normal SuccessfulCreate 6m38s statefulset-controller create Claim manager-dynamic-volume-acking-manager-0 Pod acking-manager-0 in StatefulSet acking-manager success
Normal SuccessfulCreate 6m38s statefulset-controller create Pod acking-manager-0 in StatefulSet acking-manager successful
```
#### Executor
```bash
Name: acking-manager
Namespace: default
CreationTimestamp: Tue, 03 May 2022 11:33:31 -0400
Selector: app=heron,topology=acking
Labels: app=heron
topology=acking
Annotations: <none>
Replicas: 1 desired | 1 total
Update Strategy: RollingUpdate
Partition: 0
Pods Status: 0 Running / 1 Waiting / 0 Succeeded / 0 Failed
Pod Template:
Labels: app=heron
topology=acking
Annotations: prometheus.io/port: 8080
prometheus.io/scrape: true
Containers:
manager:
Image: apache/heron:testbuild
Ports: 6001/TCP, 6002/TCP, 6003/TCP, 6004/TCP, 6005/TCP, 6006/TCP, 6007/TCP, 6008/TCP, 6009/TCP, 7775/TCP, 7776/UDP
Host Ports: 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/UDP
Command:
sh
-c
./heron-core/bin/heron-downloader-config kubernetes && ./heron-core/bin/heron-downloader distributedlog://zookeeper:2181/heronbkdl/acking-saad-tag-0--3944535742942496843.tar.gz . && SHARD_ID=${POD_NAME##*-} && echo shardId=${SHARD_ID} && ./heron-core/bin/heron-executor --topology-name=acking --topology-id=acking3ae6b6a0-bbb8-45a1-8015-37fd50fec7c6 --topology-defn-file=acking.defn --state-manager-connection=zookeeper:2181 --state-manager-root=/heron --state-manager-config-file=./heron-conf/statemgr.yaml --tmanager-binary=./heron-core/bin/heron-tmanager --stmgr-binary=./heron-core/bin/heron-stmgr --metrics-manager-classpath=./heron-core/lib/metricsmgr/* --instance-jvm-opts="LVhYOitIZWFwRHVtcE9uT3V0T2ZNZW1vcnlFcnJvcg(61)(61)" --classpath=heron-api-examples.jar --heron-internals-config-file=./heron-conf/heron_internals.yaml --override-config-file=./heron-conf/override.yaml --component-ram-map=exclaim1:1073741824,word:1073741824 --component-jvm-opts="" --pkg-type=jar --topology-b
inary-file=heron-api-examples.jar --heron-java-home=$JAVA_HOME --heron-shell-binary=./heron-core/bin/heron-shell --cluster=kubernetes --role=saad --environment=default --instance-classpath=./heron-core/lib/instance/* --metrics-sinks-config-file=./heron-conf/metrics_sinks.yaml --scheduler-classpath=./heron-core/lib/scheduler/*:./heron-core/lib/packing/*:./heron-core/lib/statemgr/* --python-instance-binary=./heron-core/bin/heron-python-instance --cpp-instance-binary=./heron-core/bin/heron-cpp-instance --metricscache-manager-classpath=./heron-core/lib/metricscachemgr/* --metricscache-manager-mode=disabled --is-stateful=false --checkpoint-manager-classpath=./heron-core/lib/ckptmgr/*:./heron-core/lib/statefulstorage/*: --stateful-config-file=./heron-conf/stateful.yaml --checkpoint-manager-ram=1073741824 --health-manager-mode=disabled --health-manager-classpath=./heron-core/lib/healthmgr/* --shard=$SHARD_ID --server-port=6001 --tmanager-controller-port=6002 --tmanager-stats-port=6003 --sh
ell-port=6004 --metrics-manager-port=6005 --scheduler-port=6006 --metricscache-manager-server-port=6007 --metricscache-manager-stats-port=6008 --checkpoint-manager-port=6009
Limits:
cpu: 2
memory: 3
Requests:
cpu: 1
memory: 2
Environment:
HOST: (v1:status.podIP)
POD_NAME: (v1:metadata.name)
var_one_manager: variable one on manager
var_three_manager: variable three on manager
var_two_manager: variable two on manager
Mounts:
/shared_volume/manager from shared-volume-manager (rw)
empty/dir/path from manager-empty-dir (ro,path="empty/dir/sub/path")
host/path/path from manager-host-path (ro,path="host/path/sub/path")
nfs/path from manager-nfs (ro,path="nfs/sub/path")
path/to/mount/dynamic/volume from manager-dynamic-volume (rw,path="sub/path/to/mount/dynamic/volume")
path/to/mount/shared/volume from manager-shared-volume (rw,path="sub/path/to/mount/shared/volume")
path/to/mount/static/volume from manager-static-volume (rw,path="sub/path/to/mount/static/volume")
manager-sidecar-container:
Image: alpine
Port: <none>
Host Port: <none>
Environment: <none>
Mounts:
/shared_volume/manager from shared-volume-manager (rw)
Volumes:
manager-empty-dir:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium: Memory
SizeLimit: 50Mi
manager-host-path:
Type: HostPath (bare host directory volume)
Path: /dev/null
HostPathType: File
manager-nfs:
Type: NFS (an NFS mount that lasts the lifetime of a pod)
Server: nfs-server.address
Path: /dev/null
ReadOnly: true
manager-shared-volume:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: requested-claim-by-user
ReadOnly: false
shared-volume-manager:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
Volume Claims:
Name: manager-static-volume
StorageClass:
Labels: onDemand=true
topology=acking
Annotations: <none>
Capacity: 512Gi
Access Modes: [ReadWriteOnce ReadOnlyMany]
Name: manager-dynamic-volume
StorageClass: storage-class-name-manager
Labels: onDemand=true
topology=acking
Annotations: <none>
Capacity: 256Gi
Access Modes: [ReadWriteOnce ReadOnlyMany]
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal SuccessfulCreate 4m46s statefulset-controller create Claim manager-static-volume-acking-manager-0 Pod acking-manager-0 in StatefulSet acking-manager success
Normal SuccessfulCreate 4m46s statefulset-controller create Claim manager-dynamic-volume-acking-manager-0 Pod acking-manager-0 in StatefulSet acking-manager success
Normal SuccessfulCreate 4m46s statefulset-controller create Pod acking-manager-0 in StatefulSet acking-manager successful
```
</details>
<details>
<summary>Persistent Volume Claims</summary>
```bash
Name: executor-dynamic-volume-acking-executor-0
Namespace: default
StorageClass: storage-class-name-executor
Status: Pending
Volume:
Labels: app=heron
onDemand=true
topology=acking
Annotations: <none>
Finalizers: [kubernetes.io/pvc-protection]
Capacity:
Access Modes:
VolumeMode: Block
Used By: acking-executor-0
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning ProvisioningFailed 2m8s (x26 over 8m9s) persistentvolume-controller storageclass.storage.k8s.io "storage-class-name-executor" not found
Name: executor-dynamic-volume-acking-executor-1
Namespace: default
StorageClass: storage-class-name-executor
Status: Pending
Volume:
Labels: app=heron
onDemand=true
topology=acking
Annotations: <none>
Finalizers: [kubernetes.io/pvc-protection]
Capacity:
Access Modes:
VolumeMode: Block
Used By: acking-executor-1
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning ProvisioningFailed 2m8s (x26 over 8m9s) persistentvolume-controller storageclass.storage.k8s.io "storage-class-name-executor" not found
Name: executor-static-volume-acking-executor-0
Namespace: default
StorageClass:
Status: Pending
Volume:
Labels: app=heron
onDemand=true
topology=acking
Annotations: <none>
Finalizers: [kubernetes.io/pvc-protection]
Capacity:
Access Modes:
VolumeMode: Block
Used By: acking-executor-0
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal FailedBinding 2m8s (x26 over 8m9s) persistentvolume-controller no persistent volumes available for this claim and no storage class is set
Name: executor-static-volume-acking-executor-1
Namespace: default
StorageClass:
Status: Pending
Volume:
Labels: app=heron
onDemand=true
topology=acking
Annotations: <none>
Finalizers: [kubernetes.io/pvc-protection]
Capacity:
Access Modes:
VolumeMode: Block
Used By: acking-executor-1
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal FailedBinding 2m8s (x26 over 8m9s) persistentvolume-controller no persistent volumes available for this claim and no storage class is set
Name: manager-dynamic-volume-acking-manager-0
Namespace: default
StorageClass: storage-class-name-manager
Status: Pending
Volume:
Labels: app=heron
onDemand=true
topology=acking
Annotations: <none>
Finalizers: [kubernetes.io/pvc-protection]
Capacity:
Access Modes:
VolumeMode: Block
Used By: acking-manager-0
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning ProvisioningFailed 2m8s (x26 over 8m9s) persistentvolume-controller storageclass.storage.k8s.io "storage-class-name-manager" not found
Name: manager-static-volume-acking-manager-0
Namespace: default
StorageClass:
Status: Pending
Volume:
Labels: app=heron
onDemand=true
topology=acking
Annotations: <none>
Finalizers: [kubernetes.io/pvc-protection]
Capacity:
Access Modes:
VolumeMode: Block
Used By: acking-manager-0
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal FailedBinding 2m8s (x26 over 8m9s) persistentvolume-controller no persistent volumes available for this claim and no storage class is set
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@heron.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org