You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by GitBox <gi...@apache.org> on 2023/01/06 21:22:19 UTC
[GitHub] [camel-karavan] mrinalsharma opened a new pull request, #592: Aws operator support
mrinalsharma opened a new pull request, #592:
URL: https://github.com/apache/camel-karavan/pull/592
This pull request is for https://github.com/apache/camel-karavan/issues/586.
- A new key `deploymentEnvironment: AWS` has been added to Karayan.yaml. This key can take values AWS, KUBERNETES, OPENSHIFT. Once Karavan.yaml is applied, Operator will update and use Key : deploymentEnvironment to decide the deployment environment.
- Added following key-values in karavan-secret.yaml
```
aws_access_key_id: aws_access_key_id
aws_secret_access_key: aws_secret_access_key
aws_region: aws_region
aws_output: json
aws_account: "aws_account"
```
- When run under AWS env, the pipeline task will create an AWS ecr repository, then create a Quarkus or Spring maven build, and then deploy in running kubernetes env.
-
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-karavan] mrinalsharma commented on pull request #592: Aws operator support
Posted by GitBox <gi...@apache.org>.
mrinalsharma commented on PR #592:
URL: https://github.com/apache/camel-karavan/pull/592#issuecomment-1379093004
Closing this Pull request. We will look into a wider use case to support different container registries and then do a rework.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-karavan] mgubaidullin commented on pull request #592: Aws operator support
Posted by GitBox <gi...@apache.org>.
mgubaidullin commented on PR #592:
URL: https://github.com/apache/camel-karavan/pull/592#issuecomment-1375902258
This is a very good idea.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-karavan] mrinalsharma commented on pull request #592: Aws operator support
Posted by GitBox <gi...@apache.org>.
mrinalsharma commented on PR #592:
URL: https://github.com/apache/camel-karavan/pull/592#issuecomment-1375791956
I am currently testing on AWS and finding out about the expiring tokens is an involved task, User will have to look at the logs of karavan-task-dev-quarkus or karavan-task-dev-spring and search for Auth errors. This will really be a very bad experience and we may see requests to add the feature to automatically create a repo and get a token.
- Do you think it will be good to give this creation of AWS repo as an option via CRD Karavan.yaml? This way if the feature is enabled then these steps to create a repo will get executed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-karavan] mrinalsharma commented on pull request #592: Aws operator support
Posted by GitBox <gi...@apache.org>.
mrinalsharma commented on PR #592:
URL: https://github.com/apache/camel-karavan/pull/592#issuecomment-1375978155
The problem is with defining AWS ecr token in Karavan.yaml. The token is very short-lived(24 hours or less) and if expired will cause the image push to fail. To troubleshoot the expired token, user will have to be aware of the internals of the working of Karavan CI/CD or somehow we should notify the user in the karavan-app UI that the token is expired. One way to handle this is that add a step to karavan-task-dev-quarkus that gets the token by logging in to docker and uses it to publish the image. This step will only get executed when running in AWS or with docker hub. The login commands will be different though.
Even to support OPENSHIFT, there are some tweaks done in the CI/CD process, I see this AWS changes the same way. Please correct me if my understanding is wrong. **_If I am understanding you correctly you don't even want to use keys like those given below in the karavan-secrets.yaml. You want to follow the same steps for all the environments and the user should take environment-specific steps by himself._**
```
aws_region: aws_region
aws_output: json
aws_account: "aws_account"
```
Summary:
What I am suggesting is this
1. Add the following keys in CRD
```
username: aws_access_key_id
password: aws_secret_access_key
aws_region: aws_region
aws_output: json
aws_account: "aws_account"
is_using_aws_ecr: true
```
2. When the user deploys the project, run a step as part of karavan-task-dev-quarkus that logs in to AWS ecr and gets the token.
3. Once the token is received, in the next step use that token as a password to push the image.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-karavan] mrinalsharma closed pull request #592: Aws operator support
Posted by GitBox <gi...@apache.org>.
mrinalsharma closed pull request #592: Aws operator support
URL: https://github.com/apache/camel-karavan/pull/592
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-karavan] mgubaidullin commented on pull request #592: Aws operator support
Posted by GitBox <gi...@apache.org>.
mgubaidullin commented on PR #592:
URL: https://github.com/apache/camel-karavan/pull/592#issuecomment-1375685730
Agree it is not simple but still. To use Karavan cloud-mode there are some prerequisites:
1. Kubernetes or OpenShift cluster
2. Tekton operator installed
3. Storage configured to be used in this Kubernetes cluster (not available by default in all cloud providers)
4. Git repository
5. Container Repository (we should not stick user with the one, ex. some AWS users set Dockerhub/Github Container registry or even Quay as a repository)
6. Infinispan installed and configured (for multi-environment or multi-cluster installation)
We can not control everything by ourselves.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-karavan] mrinalsharma commented on pull request #592: Aws operator support
Posted by GitBox <gi...@apache.org>.
mrinalsharma commented on PR #592:
URL: https://github.com/apache/camel-karavan/pull/592#issuecomment-1375879277
In this case, even the deploy step can fail from time to time because Token is short-lived, How about removing the step to create a repository but run the step to get the token automatically? This way deploy will not fail and the user will not have to troubleshoot the CI/CD process each time changes are made to integration.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-karavan] mrinalsharma commented on pull request #592: Aws operator support
Posted by GitBox <gi...@apache.org>.
mrinalsharma commented on PR #592:
URL: https://github.com/apache/camel-karavan/pull/592#issuecomment-1374957731
> 1. `karavan-secrets.yaml` might include username and password/token to access to the repository
- Token is not something pre-configured, urer needs to have a Token created by running aws-cli tool. This token also has a short life. So users will have to continuously keep updating karavan-secrets.yaml. This will be a big nuisance to users.
- If we make Container repository creation a pre-requisite step then again for each project user will have to go out of Karavan-app and create a repo.
- **The operator is not creating a Repo, the operator is just creating a karavan-task-dev-quarkus, karavan-task-dev-spring, as part of this task execution, the repo is getting created.** .
- Please suggest if you still want me to remove the task that creates the repo. Add a token to the secret?
2. Do you mean we don't need OPENSHIFT and Kubernetes keys and if AWS key is set then consider running in AWS env and use AWS properties?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-karavan] mgubaidullin commented on pull request #592: Aws operator support
Posted by GitBox <gi...@apache.org>.
mgubaidullin commented on PR #592:
URL: https://github.com/apache/camel-karavan/pull/592#issuecomment-1376175268
Lets take a look on the widely used cases and define parameters we need to support these cases:
https://docs.google.com/spreadsheets/d/1Vc9Hnti_kB7LOp7mqcYTliNqnJiiWCHeOeUZOk4CHSA
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-karavan] mgubaidullin commented on pull request #592: Aws operator support
Posted by GitBox <gi...@apache.org>.
mgubaidullin commented on PR #592:
URL: https://github.com/apache/camel-karavan/pull/592#issuecomment-1374875299
1. Operator should not create anything outside Kubernetes. Created Container repository (ECR or other) should be a documented prerequisite. `karavan-secrets.yaml` might include username and password/token to access to the repository
2. Both Kubernetes and OpenShift might be installed in AWS, Azure, GCP. Cloud vendor provided Kubernetes aka AKS, EKS, GKE as far as I know are pure Kubernetes, so they have same Kubernetes API.
3. PVC StorageClassName and AccessModes are generic for any kind of installation and might be fields in Karavan CRD
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-karavan] mgubaidullin commented on pull request #592: Aws operator support
Posted by GitBox <gi...@apache.org>.
mgubaidullin commented on PR #592:
URL: https://github.com/apache/camel-karavan/pull/592#issuecomment-1374876778
Related to this https://github.com/apache/camel-karavan/issues/512
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-karavan] mgubaidullin commented on pull request #592: Aws operator support
Posted by GitBox <gi...@apache.org>.
mgubaidullin commented on PR #592:
URL: https://github.com/apache/camel-karavan/pull/592#issuecomment-1375846494
Tekton pipeline is CI/CD process to compile code, package jar, create and push image, deploy application.
Repository creation is not a part of CI/CD process.
Yes, it makes easy to start with app but does not make it right.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-karavan] mgubaidullin commented on pull request #592: Aws operator support
Posted by GitBox <gi...@apache.org>.
mgubaidullin commented on PR #592:
URL: https://github.com/apache/camel-karavan/pull/592#issuecomment-1376175787
<img width="1318" alt="Screenshot 2023-01-09 at 2 33 09 PM" src="https://user-images.githubusercontent.com/1379213/211392417-a94000e9-0304-421a-a70d-c32a395456b1.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org