You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@devlake.apache.org by "DarkoDevops (via GitHub)" <gi...@apache.org> on 2023/05/15 12:12:22 UTC
[GitHub] [incubator-devlake-helm-chart] DarkoDevops opened a new issue, #131: Helm Chart installation on AWS EKS: Unable to attach or mount volumes
DarkoDevops opened a new issue, #131:
URL: https://github.com/apache/incubator-devlake-helm-chart/issues/131
I installed DevLake using Helm chart on my AWS EKS cluster. Used previous instructions from ticket:
https://github.com/apache/incubator-devlake-helm-chart/issues/87 but, without success. Still not working.
Helm values.yaml looks like:
```
replicaCount: 1
imageTag: v0.16.0-beta17
mysql:
useExternal: false
externalServer: 127.0.0.1
externalPort: 3306
username: merico
password: merico
database: lake
rootPassword: admin
storage:
class: "gp2"
size: 50Gi
image:
repository: mysql
tag: 8
pullPolicy: IfNotPresent
resources: {}
nodeSelector: {}
tolerations: []
affinity: {}
grafana:
image:
repository: apache/devlake-dashboard
pullPolicy: Always
useExternal: false
externalUrl: ''
resources: {}
nodeSelector: {}
tolerations: []
affinity: {}
lake:
image:
repository: apache/devlake
pullPolicy: Always
storage:
class: "gp2"
size: 100Mi
dotenv:
API_TIMEOUT: "120s"
API_RETRY: "3"
API_REQUESTS_PER_HOUR: "10000"
PIPELINE_MAX_PARALLEL: "1"
IN_SECURE_SKIP_VERIFY: "false"
hostNetwork: false
resources: {}
nodeSelector: {}
tolerations: []
affinity: {}
loggingDir: "/app/logs"
loggingLevel: "info"
ui:
image:
repository: apache/devlake-config-ui
pullPolicy: Always
resources: {}
nodeSelector: {}
tolerations: []
affinity: {}
basicAuth:
enabled: false
user: admin
password: admin
useSecret: false
autoCreateSecret: true
secretName: devlake-auth
alpine:
image:
repository: alpine
tag: 3.16
pullPolicy: IfNotPresent
service:
type: NodePort
uiPort: 32001
grafanaPort : 32002
ingress:
enabled: true
enableHttps: false
className: "alb"
prefix: /
tlsSecretName: ""
httpPort: 80
httpsPort: 443
useDefaultNginx: false
annotations:
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/target-type: ip
option:
localtime: /etc/localtime
database: mysql
useConnectionDetailsSecret: false
connectionSecretName: devlake-db-connection
autoCreateSecret: true
```
I get multiple errors like:
```
Unable to attach or mount volumes: unmounted volumes=[devlake-lake-config], unattached volumes=[devlake-lake-localtime kube-api-access-r7dgn devlake-lake-config]: timed out waiting for the condition
Unable to attach or mount volumes: unmounted volumes=[devlake-lake-config], unattached volumes=[kube-api-access-r7dgn devlake-lake-config devlake-lake-localtime]: timed out waiting for the condition
Unable to attach or mount volumes: unmounted volumes=[devlake-lake-config], unattached volumes=[devlake-lake-config devlake-lake-localtime kube-api-access-r7dgn]: timed out waiting for the condition
AttachVolume.Attach failed for volume "pvc-09b75f09-f0e4-4700-b42c-e9f901a13bf2" : volume attachment is being deleted
MountVolume.WaitForAttach failed for volume "pvc-09b75f09-f0e4-4700-b42c-e9f901a13bf2" : volume attachment is being deleted
```
Also tried to set storage class to empty (default) and same behaviour. Any idea?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake-helm-chart] DarkoDevops commented on issue #131: Helm Chart installation on AWS EKS: Unable to attach or mount volumes
Posted by "DarkoDevops (via GitHub)" <gi...@apache.org>.
DarkoDevops commented on issue #131:
URL: https://github.com/apache/incubator-devlake-helm-chart/issues/131#issuecomment-1552906498
This is what I have of the pods for ebs:
```
ebs-csi-controller-78bdbf98ff-574px 5/5 Running 0 2d5h
ebs-csi-controller-78bdbf98ff-hwktx 5/5 Running 2 (8d ago) 8d
ebs-csi-node-46nh7 3/3 Running 0 8d
ebs-csi-node-dthdq 3/3 Running 0 2d5h
ebs-csi-node-fpz98 3/3 Running 0 8d
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake-helm-chart] BrookeKatalon commented on issue #131: Helm Chart installation on AWS EKS: Unable to attach or mount volumes
Posted by "BrookeKatalon (via GitHub)" <gi...@apache.org>.
BrookeKatalon commented on issue #131:
URL: https://github.com/apache/incubator-devlake-helm-chart/issues/131#issuecomment-1557648510
Seeing the same issue with imageTag: v0.17.0-beta5
containers have errors like:
`MountVolume.SetUp failed for volume "devlake-mysql-localtime" : hostPath type check failed: /etc/localtime is not a file`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake-helm-chart] JorgeGar commented on issue #131: Helm Chart installation on AWS EKS: Unable to attach or mount volumes
Posted by "JorgeGar (via GitHub)" <gi...@apache.org>.
JorgeGar commented on issue #131:
URL: https://github.com/apache/incubator-devlake-helm-chart/issues/131#issuecomment-1552713249
I'm running similar values without any issues. I have mysql in an external database, but that's not the one giving you issues, as per what I see in your comment @DarkoDevops. The key difference I guess is that I have aws ebs csi driver installed in the cluster, rather than it being an add-on. Could you try and get logs from it?
Also, which EKS version are you using?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake-helm-chart] DarkoDevops commented on issue #131: Helm Chart installation on AWS EKS: Unable to attach or mount volumes
Posted by "DarkoDevops (via GitHub)" <gi...@apache.org>.
DarkoDevops commented on issue #131:
URL: https://github.com/apache/incubator-devlake-helm-chart/issues/131#issuecomment-1552581051
@IronCore864 I will run it and post it.
Yes, I have aws-ebs-csi-driver addon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake-helm-chart] JorgeGar commented on issue #131: Helm Chart installation on AWS EKS: Unable to attach or mount volumes
Posted by "JorgeGar (via GitHub)" <gi...@apache.org>.
JorgeGar commented on issue #131:
URL: https://github.com/apache/incubator-devlake-helm-chart/issues/131#issuecomment-1552901754
> @JorgeGar It's 1.25
>
> It's installed, in cluster, also, added policy, not the add-on.
Have you checked the logs of the attacher?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake-helm-chart] DarkoDevops commented on issue #131: Helm Chart installation on AWS EKS: Unable to attach or mount volumes
Posted by "DarkoDevops (via GitHub)" <gi...@apache.org>.
DarkoDevops commented on issue #131:
URL: https://github.com/apache/incubator-devlake-helm-chart/issues/131#issuecomment-1552895816
@JorgeGar
It's 1.25
It's installed, in cluster, also, added policy, not the add-on.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake-helm-chart] IronCore864 commented on issue #131: Helm Chart installation on AWS EKS: Unable to attach or mount volumes
Posted by "IronCore864 (via GitHub)" <gi...@apache.org>.
IronCore864 commented on issue #131:
URL: https://github.com/apache/incubator-devlake-helm-chart/issues/131#issuecomment-1552564847
@DarkoDevops can you run kubectl get/describe pv/pvc and see what you can get from the output?
A wild guess: do you have the aws-ebs-csi-driver addon enabled in your EKS?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake-helm-chart] DarkoDevops commented on issue #131: Helm Chart installation on AWS EKS: Unable to attach or mount volumes
Posted by "DarkoDevops (via GitHub)" <gi...@apache.org>.
DarkoDevops commented on issue #131:
URL: https://github.com/apache/incubator-devlake-helm-chart/issues/131#issuecomment-1552677445
pv/pvc describe:
```
pvc-09b75f09-f0e4-4700-b42c-e9f901a13bf2 1Gi RWO Delete Bound default/devlake-lake-config-devlake-lake-0 gp2 65d
pvc-16438fe5-4163-441b-be46-2a9d8920b99f 50Gi RWO Delete Bound default/devlake-mysql-data-devlake-mysql-0 gp2 65d
```
```
Name: pvc-09b75f09-f0e4-4700-b42c-e9f901a13bf2
Labels: topology.kubernetes.io/region=eu-central-1
topology.kubernetes.io/zone=eu-central-1a
Annotations: kubernetes.io/createdby: aws-ebs-dynamic-provisioner
pv.kubernetes.io/bound-by-controller: yes
pv.kubernetes.io/migrated-to: ebs.csi.aws.com
pv.kubernetes.io/provisioned-by: kubernetes.io/aws-ebs
Finalizers: [kubernetes.io/pv-protection external-attacher/ebs-csi-aws-com]
StorageClass: gp2
Status: Bound
Claim: default/devlake-lake-config-devlake-lake-0
Reclaim Policy: Delete
Access Modes: RWO
VolumeMode: Filesystem
Capacity: 1Gi
Node Affinity:
Required Terms:
Term 0: topology.kubernetes.io/zone in [eu-central-1a]
topology.kubernetes.io/region in [eu-central-1]
Message:
Source:
Type: AWSElasticBlockStore (a Persistent Disk resource in AWS)
VolumeID: aws://eu-central-1a/vol-0b203af215cc352d9
FSType: ext4
Partition: 0
ReadOnly: false
Events: <none>
```
```
Name: pvc-16438fe5-4163-441b-be46-2a9d8920b99f
Labels: topology.kubernetes.io/region=eu-central-1
topology.kubernetes.io/zone=eu-central-1b
Annotations: kubernetes.io/createdby: aws-ebs-dynamic-provisioner
pv.kubernetes.io/bound-by-controller: yes
pv.kubernetes.io/migrated-to: ebs.csi.aws.com
pv.kubernetes.io/provisioned-by: kubernetes.io/aws-ebs
Finalizers: [kubernetes.io/pv-protection external-attacher/ebs-csi-aws-com]
StorageClass: gp2
Status: Bound
Claim: default/devlake-mysql-data-devlake-mysql-0
Reclaim Policy: Delete
Access Modes: RWO
VolumeMode: Filesystem
Capacity: 50Gi
Node Affinity:
Required Terms:
Term 0: topology.kubernetes.io/zone in [eu-central-1b]
topology.kubernetes.io/region in [eu-central-1]
Message:
Source:
Type: AWSElasticBlockStore (a Persistent Disk resource in AWS)
VolumeID: aws://eu-central-1b/vol-07a37b8cc48ea301f
FSType: ext4
Partition: 0
ReadOnly: false
Events: <none>
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake-helm-chart] JorgeGar commented on issue #131: Helm Chart installation on AWS EKS: Unable to attach or mount volumes
Posted by "JorgeGar (via GitHub)" <gi...@apache.org>.
JorgeGar commented on issue #131:
URL: https://github.com/apache/incubator-devlake-helm-chart/issues/131#issuecomment-1552911328
Honestly, at this point, I'm following the [troubleshooting page from AWS](https://repost.aws/knowledge-center/eks-troubleshoot-ebs-volume-mounts), but...
On the ebs-csi-controller, you should see a container called "csi-attacher". Try getting logs from there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake-helm-chart] klesh commented on issue #131: Helm Chart installation on AWS EKS: Unable to attach or mount volumes
Posted by "klesh (via GitHub)" <gi...@apache.org>.
klesh commented on issue #131:
URL: https://github.com/apache/incubator-devlake-helm-chart/issues/131#issuecomment-1552531884
@IronCore864 @JorgeGar would you mind take a look at this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org