[jira] [Created] (FLINK-29110) Support to mount a dynamically-created pvc for JM and TM in standalone mode with StatefulSet.

["Peng Yuan (Jira)" <ji...@apache.org>]

Peng Yuan created FLINK-29110:
---------------------------------

             Summary: Support to mount a dynamically-created pvc for JM and TM in standalone mode with StatefulSet.
                 Key: FLINK-29110
                 URL: https://issues.apache.org/jira/browse/FLINK-29110
             Project: Flink
          Issue Type: New Feature
          Components: Deployment / Kubernetes
            Reporter: Peng Yuan


Use StatefulSet instead of Deployment to deploy JM and TM to support mount a dynamically-created PersistentVolumeClaim.

add volumeClaimTemplates to JobManagerSpec and TaskManagerSpec:

JobManagerSpec:

 
{code:java}
public class JobManagerSpec {
    /** Resource specification for the JobManager pods. */
    private Resource resource;

    /** Number of JobManager replicas. Must be 1 for non-HA deployments. */
    private int replicas = 1;

    /** Volume Claim Templates for JobManager stateful set. Just for standalone mode. */
    private List<PersistentVolumeClaim> volumeClaimTemplates = new ArrayList<>();

    /** JobManager pod template. It will be merged with FlinkDeploymentSpec.podTemplate. */
    private Pod podTemplate;
}
 {code}
TaskManagerSpec:

 
{code:java}
public class TaskManagerSpec {
    /** Resource specification for the TaskManager pods. */
    private Resource resource;

    /** Number of TaskManager replicas. If defined, takes precedence over parallelism */
    @SpecReplicas private Integer replicas;

    /** Volume Claim Templates for TaskManager stateful set. Just for standalone mode. */
    private List<PersistentVolumeClaim> volumeClaimTemplates = new ArrayList<>();

    /** TaskManager pod template. It will be merged with FlinkDeploymentSpec.podTemplate. */
    private Pod podTemplate;
} {code}
 

volumeClaimTemplates just available in standalone mode.

 

CR Example:

 
{code:java}
kind: FlinkDeployment
metadata:
  namespace: default
  name: basic-example
spec:
  image: flink:1.14.3
  flinkVersion: v1_14
  flinkConfiguration:
    taskmanager.numberOfTaskSlots: "2"
  serviceAccount: flink
  jobManager:
    replicas: 1
    resource:
      memory: "2048m"
      cpu: 1
    volumeClaimTemplates:
      - metadata:
          name: log
        spec:
          accessModes: [ "ReadWriteOnce" ]
          storageClassName: "alicloud-local-lvm"
          resources:
            requests:
              storage: 10Gi
    podTemplate:
      apiVersion: v1
      kind: Pod
      metadata:
        name: job-manager-pod-template
      spec:
        containers:
          - name: flink-main-container
            volumeMounts:
              - name: log-volume
                mountPath: /opt/flink/log
  taskManager:
    resource:
      replicas: 1 // (only needed for standalone clusters)*
      memory: "2048m"
      cpu: 1
    volumeClaimTemplates:
      - metadata:
          name: log
        spec:
          accessModes: [ "ReadWriteOnce" ]
          storageClassName: "alicloud-local-lvm"
          resources:
            requests:
              storage: 10Gi
    podTemplate:
      apiVersion: v1
      kind: Pod
      metadata:
        name: task-manager-pod-template
      spec:
        containers:
          - name: flink-main-container
            volumeMounts:
              - name: log-volume
                mountPath: /opt/flink/log
  mode: standalone {code}
 

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)