You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by "soulmem (via GitHub)" <gi...@apache.org> on 2025/09/10 05:07:04 UTC
[I] [Helm / K8s] 3.3.1 fresh install only works after manually copying storage JARs and changing ConfigMap mount path – is this expected? [dolphinscheduler]
soulmem opened a new issue, #17495:
URL: https://github.com/apache/dolphinscheduler/issues/17495
### Search before asking
- [x] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues.
### What happened
A clean install fails until two manual changes are made:
1.Missing storage plug-in JARs
The storage JARs (dolphinscheduler-storage-s3-*.jar, dolphinscheduler-storage-hdfs-*.jar, etc.) exist in /opt/dolphinscheduler/libs inside the official images, but are NOT present in /opt/dolphinscheduler/worker-server/libs or /opt/dolphinscheduler/api-server/libs.
→ I had to copy them into those sub-folders and re-build custom images; otherwise Worker and API pods crash with No qualifying bean of type 'org.apache.dolphinscheduler.plugin.storage.api.StorageOperator'.
2.Wrong ConfigMap mount path
statefulset-dolphinscheduler-worker.yaml and deployment-dolphinscheduler-api.yaml mount config-volume at
/opt/dolphinscheduler/conf/common.properties.
However, the start-scripts inside worker-server and api-server actually load the file from
/opt/dolphinscheduler/worker-server/conf/common.properties and /opt/dolphinscheduler/api-server/conf/common.properties respectively.
→ I had to patch both manifests and move the mount path, otherwise storage-type is ignored and the same “StorageOperator not found” error appears.
After these two work-arounds the cluster starts and workflows run correctly.
### What you expected to happen
May I ask, although everything is working now, could these manual changes I made introduce any hidden risks or future operational problems?
### How to reproduce
This is my values.yaml:
alert:
affinity: {}
annotations: {}
customizedConfig: {}
enableCustomizedConfig: false
enabled: true
env:
JAVA_OPTS: '-Xms512m -Xmx512m -Xmn256m'
livenessProbe:
enabled: true
failureThreshold: '3'
initialDelaySeconds: '30'
periodSeconds: '30'
successThreshold: '1'
timeoutSeconds: '5'
nodeSelector: {}
persistentVolumeClaim:
accessModes:
- ReadWriteOnce
enabled: true
storage: 20Gi
storageClassName: vsphere-csi-sc
readinessProbe:
enabled: true
failureThreshold: '3'
initialDelaySeconds: '30'
periodSeconds: '30'
successThreshold: '1'
timeoutSeconds: '5'
replicas: 1
resources: {}
service:
annotations: {}
serviceMonitor:
annotations: {}
enabled: false
interval: 15s
labels: {}
path: /actuator/prometheus
strategy:
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
type: RollingUpdate
tolerations: []
api:
affinity: {}
annotations: {}
customizedConfig: {}
enableCustomizedConfig: false
enabled: true
env:
JAVA_OPTS: '-Xms512m -Xmx512m -Xmn256m'
livenessProbe:
enabled: true
failureThreshold: '3'
initialDelaySeconds: '30'
periodSeconds: '30'
successThreshold: '1'
timeoutSeconds: '5'
nodeSelector: {}
persistentVolumeClaim:
accessModes:
- ReadWriteOnce
enabled: true
storage: 20Gi
storageClassName: vsphere-csi-sc
readinessProbe:
enabled: true
failureThreshold: '3'
initialDelaySeconds: '30'
periodSeconds: '30'
successThreshold: '1'
timeoutSeconds: '5'
replicas: '1'
resources: {}
service:
annotations: {}
clusterIP: ''
externalIPs: []
externalName: ''
loadBalancerIP: ''
nodePort: ''
pythonNodePort: ''
serviceMonitor:
annotations: {}
enabled: false
interval: 15s
labels: {}
path: /dolphinscheduler/actuator/prometheus
type: ClusterIP
strategy:
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
type: RollingUpdate
taskTypeFilter:
enabled: false
task: {}
tolerations: []
common:
configmap:
DATAX_LAUNCHER: /opt/soft/datax/bin/datax.py
DATA_BASEDIR_PATH: /tmp/dolphinscheduler
DOLPHINSCHEDULER_OPTS: ''
FLINK_HOME: /opt/soft/flink
HADOOP_CONF_DIR: /opt/soft/hadoop/etc/hadoop
HADOOP_HOME: /opt/soft/hadoop
HIVE_HOME: /opt/soft/hive
JAVA_HOME: /opt/java/openjdk
PYTHON_LAUNCHER: /usr/bin/python3
RESOURCE_UPLOAD_PATH: /dolphinscheduler
SPARK_HOME: /opt/soft/spark
RESOURCE_STORAGE_TYPE: S3
fsFileResourcePersistence:
accessModes:
- ReadWriteMany
enabled: true
storage: 20Gi
storageClassName: vsphere-csi-sc-rwx
sharedStoragePersistence:
accessModes:
- ReadWriteMany
enabled: true
mountPath: /opt/soft
storage: 20Gi
storageClassName: vsphere-csi-sc-rwx
conf:
auto: false
common:
appId.collect: log
aws.credentials.provider.type: AWSStaticCredentialsProvider
aws.s3.access.key.id: dolphinscheduler
aws.s3.access.key.secret: Admin@cdu1stc.cn
aws.s3.bucket.name: dolphinscheduler
aws.s3.endpoint: https://oss.cduestc.cn:443
aws.s3.region: ca-central-1
conda.path: /opt/anaconda3/etc/profile.d/conda.sh
data.basedir.path: /tmp/dolphinscheduler
datasource.encryption.enable: false
datasource.encryption.salt: '!@#$%^&*'
development.state: false
hadoop.security.authentication.startup.state: false
java.security.krb5.conf.path: /opt/krb5.conf
kerberos.expire.time: 2
login.user.keytab.path: /opt/hdfs.headless.keytab
login.user.keytab.username: hdfs-mycluster@ESZ.COM
ml.mlflow.preset_repository: https://github.com/apache/dolphinscheduler-mlflow
ml.mlflow.preset_repository_version: main
resource.aws.credentials.provider.type: AWSStaticCredentialsProvider
resource.aws.s3.access.key.id: dolphinscheduler
resource.aws.s3.access.key.secret: Admin@cdu1stc.cn
resource.aws.s3.bucket.name: dolphinscheduler
resource.aws.s3.endpoint: https://oss.cduestc.cn:443
resource.aws.s3.region: ca-central-1
resource.alibaba.cloud.access.key.id: <your-access-key-id>
resource.alibaba.cloud.access.key.secret: <your-access-key-secret>
resource.alibaba.cloud.oss.bucket.name: dolphinscheduler
resource.alibaba.cloud.oss.endpoint: https://oss-cn-hangzhou.aliyuncs.com
resource.alibaba.cloud.region: cn-hangzhou
resource.azure.client.id: minioadmin
resource.azure.client.secret: minioadmin
resource.azure.subId: minioadmin
resource.azure.tenant.id: minioadmin
resource.hdfs.fs.defaultFS: hdfs://mycluster:8020
resource.hdfs.root.user: hdfs
resource.manager.httpaddress.port: 8088
resource.storage.type: S3
resource.storage.upload.base.path: /dolphinscheduler
shell.kill.wait.timeout: 10
sudo.enable: true
support.hive.oneSession: false
task.resource.limit.state: false
yarn.application.status.address: http://ds1:%s/ws/v1/cluster/apps/%s
yarn.job.history.status.address: http://ds1:19888/ws/v1/history/mapreduce/jobs/%s
yarn.resourcemanager.ha.rm.ids: 192.168.xx.xx,192.168.xx.xx
datasource:
profile: postgresql
externalDatabase:
database: dolphinscheduler
driverClassName: org.postgresql.Driver
enabled: false
host: localhost
params: characterEncoding=utf8
password: root
port: '5432'
type: postgresql
username: root
externalRegistry:
registryPluginName: zookeeper
registryServers: zookeeper-headless.zookeeper.svc.cluster.local:2181
image:
alert: dolphinscheduler-alert-server
api: dolphinscheduler-api
master: dolphinscheduler-master
pullPolicy: IfNotPresent
pullSecret: cduestc-harbor
registry: myharbor/library
tag: 3.3.1
tools: dolphinscheduler-tools
worker: dolphinscheduler-worker
ingress:
annotations: {}
enabled: true
host: xxx.mydomain.cn
path: /dolphinscheduler
tls:
enabled: false
secretName: dolphinscheduler-tls
initImage:
busybox: busybox:1.30.1
pullPolicy: IfNotPresent
master:
affinity: {}
annotations: {}
customizedConfig: {}
enableCustomizedConfig: false
enabled: true
env:
JAVA_OPTS: '-Xms1g -Xmx1g -Xmn512m'
MASTER_DISPATCH_TASK_NUM: '3'
MASTER_EXEC_TASK_NUM: '20'
MASTER_EXEC_THREADS: '100'
MASTER_FAILOVER_INTERVAL: 10m
MASTER_HEARTBEAT_ERROR_THRESHOLD: '5'
MASTER_HOST_SELECTOR: LowerWeight
MASTER_KILL_APPLICATION_WHEN_HANDLE_FAILOVER: 'true'
MASTER_MAX_HEARTBEAT_INTERVAL: 10s
MASTER_SERVER_LOAD_PROTECTION_ENABLED: false
MASTER_SERVER_LOAD_PROTECTION_MAX_DISK_USAGE_PERCENTAGE_THRESHOLDS: 0.7
MASTER_SERVER_LOAD_PROTECTION_MAX_JVM_CPU_USAGE_PERCENTAGE_THRESHOLDS: 0.7
MASTER_SERVER_LOAD_PROTECTION_MAX_SYSTEM_CPU_USAGE_PERCENTAGE_THRESHOLDS: 0.7
MASTER_SERVER_LOAD_PROTECTION_MAX_SYSTEM_MEMORY_USAGE_PERCENTAGE_THRESHOLDS: 0.7
MASTER_STATE_WHEEL_INTERVAL: 5s
MASTER_TASK_COMMIT_INTERVAL: 1s
MASTER_TASK_COMMIT_RETRYTIMES: '5'
livenessProbe:
enabled: true
failureThreshold: '3'
initialDelaySeconds: '30'
periodSeconds: '30'
successThreshold: '1'
timeoutSeconds: '5'
nodeSelector: {}
persistentVolumeClaim:
accessModes:
- ReadWriteOnce
enabled: true
storage: 20Gi
storageClassName: vsphere-csi-sc
podManagementPolicy: Parallel
readinessProbe:
enabled: true
failureThreshold: '3'
initialDelaySeconds: '30'
periodSeconds: '30'
successThreshold: '1'
timeoutSeconds: '5'
replicas: '3'
resources: {}
service:
annotations: {}
serviceMonitor:
annotations: {}
enabled: false
interval: 15s
labels: {}
path: /actuator/prometheus
tolerations: []
updateStrategy:
type: RollingUpdate
minio:
auth:
rootPassword: Admin@cdu1stc.cn
rootUser: minioadmin
defaultBuckets: dolphinscheduler
enabled: false
persistence:
enabled: false
mysql:
auth:
database: dolphinscheduler
params: characterEncoding=utf8
password: ds
username: ds
driverClassName: com.mysql.cj.jdbc.Driver
enabled: false
primary:
persistence:
enabled: false
size: 20Gi
storageClass: vsphere-csi-sc
postgresql:
driverClassName: org.postgresql.Driver
enabled: true
params: characterEncoding=utf8
persistence:
enabled: true
size: 100Gi
storageClass: vsphere-csi-sc
postgresqlDatabase: dolphinscheduler
postgresqlPassword: Admin@cdu1stc.cn
postgresqlUsername: root
registryEtcd:
authority: ''
enabled: false
endpoints: ''
namespace: dolphinscheduler
passWord: ''
ssl:
certFile: etcd-certs/ca.crt
enabled: false
keyCertChainFile: etcd-certs/client.crt
keyFile: etcd-certs/client.pem
user: ''
registryJdbc:
enabled: false
hikariConfig:
driverClassName: com.mysql.cj.jdbc.Driver
enabled: false
jdbcurl: jdbc:mysql://
password: ''
username: ''
termExpireTimes: 3
termRefreshInterval: 2s
security:
authentication:
ldap:
basedn: dc=example,dc=com
password: password
ssl:
enable: false
jksbase64content: ''
truststore: /opt/ldapkeystore.jks
truststorepassword: ''
urls: ldap://ldap.forumsys.com:389/
user:
admin: read-only-admin
emailattribute: mail
identityattribute: uid
notexistaction: CREATE
username: cn=read-only-admin,dc=example,dc=com
type: PASSWORD
timezone: Asia/Shanghai
worker:
affinity: {}
annotations: {}
customizedConfig: {}
enableCustomizedConfig: false
enabled: true
env:
WORKER_EXEC_THREADS: '100'
WORKER_HOST_WEIGHT: '100'
WORKER_MAX_HEARTBEAT_INTERVAL: 10s
WORKER_SERVER_LOAD_PROTECTION_ENABLED: false
WORKER_SERVER_LOAD_PROTECTION_MAX_DISK_USAGE_PERCENTAGE_THRESHOLDS: 0.7
WORKER_SERVER_LOAD_PROTECTION_MAX_JVM_CPU_USAGE_PERCENTAGE_THRESHOLDS: 0.7
WORKER_SERVER_LOAD_PROTECTION_MAX_SYSTEM_CPU_USAGE_PERCENTAGE_THRESHOLDS: 0.7
WORKER_SERVER_LOAD_PROTECTION_MAX_SYSTEM_MEMORY_USAGE_PERCENTAGE_THRESHOLDS: 0.7
WORKER_TENANT_CONFIG_AUTO_CREATE_TENANT_ENABLED: true
WORKER_TENANT_CONFIG_DEFAULT_TENANT_ENABLED: false
keda:
advanced: {}
cooldownPeriod: 30
enabled: false
maxReplicaCount: 3
minReplicaCount: 0
namespaceLabels: {}
pollingInterval: 5
livenessProbe:
enabled: true
failureThreshold: '3'
initialDelaySeconds: '30'
periodSeconds: '30'
successThreshold: '1'
timeoutSeconds: '5'
nodeSelector: {}
persistentVolumeClaim:
dataPersistentVolume:
accessModes:
- ReadWriteOnce
enabled: true
storage: 20Gi
storageClassName: vsphere-csi-sc
enabled: false
logsPersistentVolume:
accessModes:
- ReadWriteOnce
enabled: true
storage: 20Gi
storageClassName: vsphere-csi-sc
podManagementPolicy: Parallel
readinessProbe:
enabled: true
failureThreshold: '3'
initialDelaySeconds: '30'
periodSeconds: '30'
successThreshold: '1'
timeoutSeconds: '5'
replicas: '3'
resources: {}
service:
annotations: {}
serviceMonitor:
annotations: {}
enabled: false
interval: 15s
labels: {}
path: /actuator/prometheus
tolerations: []
updateStrategy:
type: RollingUpdate
extraVolumeMounts:
- name: worker-cm
mountPath: /opt/dolphinscheduler/worker-server/conf/common.properties
subPath: common.properties
extraVolumes:
- name: worker-cm
configMap:
name: ds-configs
zookeeper:
enabled: false
fourlwCommandsWhitelist: srvr,ruok,wchs,cons
persistence:
enabled: false
size: 20Gi
storageClass: vsphere-csi-sc
service:
port: 2181
global:
cattle:
systemProjectId: p-np6fz
### Anything else
_No response_
### Version
dev
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Helm / K8s] 3.3.1 fresh install only works after manually copying storage JARs and changing ConfigMap mount path – is this expected? [dolphinscheduler]
Posted by "SbloodyS (via GitHub)" <gi...@apache.org>.
SbloodyS closed issue #17495: [Helm / K8s] 3.3.1 fresh install only works after manually copying storage JARs and changing ConfigMap mount path – is this expected?
URL: https://github.com/apache/dolphinscheduler/issues/17495
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Helm / K8s] 3.3.1 fresh install only works after manually copying storage JARs and changing ConfigMap mount path – is this expected? [dolphinscheduler]
Posted by "soulmem (via GitHub)" <gi...@apache.org>.
soulmem commented on issue #17495:
URL: https://github.com/apache/dolphinscheduler/issues/17495#issuecomment-3315552014
The `install-plugin` script can add missing JARs to the `plugins` directory, but I found that I still need to manually create symbolic links to the `libs` directories under `api`, `master`, and `worker` (as well as the configuration files under `conf`). Could this operation be documented somewhere so that it’s easier for others to find in the future? Or if such documentation already exists and I just missed it, then please disregard this suggestion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Helm / K8s] 3.3.1 fresh install only works after manually copying storage JARs and changing ConfigMap mount path – is this expected? [dolphinscheduler]
Posted by "SebastienTolron (via GitHub)" <gi...@apache.org>.
SebastienTolron commented on issue #17495:
URL: https://github.com/apache/dolphinscheduler/issues/17495#issuecomment-3361124436
Thanks a lot for the instructions on how to make the helm chart working !
This is not normal. Helm deployment should work out of the box
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org