You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2022/08/24 07:42:50 UTC
[GitHub] [dolphinscheduler] 8431 opened a new issue, #11627: [Bug] [dolphinscheduler-common] When an application is deployed on the EC2 machine on AWS and S3 is operated, it will be rejected
8431 opened a new issue, #11627:
URL: https://github.com/apache/dolphinscheduler/issues/11627
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues.
### What happened
When you deploy an application on the EC2 machine of AWS to operate S3, it will be rejected
AWS EC2 machine will not use key access because of security policy
### What you expected to happen
When you deploy an application on the EC2 machine of AWS to operate S3, it will be rejected
AWS EC2 machine will not use key access because of security policy
Therefore, this problem can be compatible when building S3 client objects, so that EC2 machines in AWS can access S3
Whether the code can be compatible with the above scenario
`AWSCredentialsProvider provider = new AWSCredentialsProviderChain(
new EC2ContainerCredentialsProviderWrapper(),
new AWSStaticCredentialsProvider(new BasicAWSCredentials(ACCESS_KEY_ID, SECRET_KEY_ID)));`
`private S3Utils() {
if (PropertyUtils.getString(RESOURCE_STORAGE_TYPE).equals(STORAGE_S3)) {
AWSCredentialsProvider provider = new AWSCredentialsProviderChain(
new EC2ContainerCredentialsProviderWrapper(),
new AWSStaticCredentialsProvider(new BasicAWSCredentials(ACCESS_KEY_ID, SECRET_KEY_ID)));
if (!StringUtils.isEmpty(PropertyUtils.getString(AWS_END_POINT))) {
s3Client = AmazonS3ClientBuilder
.standard()
.withPathStyleAccessEnabled(true)
.withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration(PropertyUtils.getString(AWS_END_POINT), Regions.fromName(REGION).getName()))
.withCredentials(provider)
.build();
} else {
s3Client = AmazonS3ClientBuilder
.standard()
.withCredentials(provider)
.withRegion(Regions.fromName(REGION))
.build();
}
checkBucketNameIfNotPresent(BUCKET_NAME);
}
}`
### How to reproduce
Deploy the dolphin scheduler project to the EC2 machine of AWS and configure S3 information. Then go to the resource management interface to create a folder
### Anything else
_No response_
### Version
3.0.0
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] 8431 commented on issue #11627: [Bug] [dolphinscheduler-common] When an application is deployed on the EC2 machine on AWS and S3 is operated, it will be rejected
Posted by GitBox <gi...@apache.org>.
8431 commented on issue #11627:
URL: https://github.com/apache/dolphinscheduler/issues/11627#issuecomment-1237581959
private S3Utils() {
if (PropertyUtils.getString(RESOURCE_STORAGE_TYPE).equals(STORAGE_S3)) {
AWSCredentialsProvider provider = new AWSCredentialsProviderChain(
new EC2ContainerCredentialsProviderWrapper(),
new AWSStaticCredentialsProvider(new BasicAWSCredentials(ACCESS_KEY_ID, SECRET_KEY_ID)));
if (!StringUtils.isEmpty(PropertyUtils.getString(AWS_END_POINT))) {
s3Client = AmazonS3ClientBuilder
.standard()
.withPathStyleAccessEnabled(true)
.withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration(PropertyUtils.getString(AWS_END_POINT), Regions.fromName(REGION).getName()))
.withCredentials(provider)
.build();
} else {
s3Client = AmazonS3ClientBuilder
.standard()
.withCredentials(provider)
.withRegion(Regions.fromName(REGION))
.build();
}
checkBucketNameIfNotPresent(BUCKET_NAME);
}
}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] 8431 commented on issue #11627: [Bug] [dolphinscheduler-common] When an application is deployed on the EC2 machine on AWS and S3 is operated, it will be rejected
Posted by GitBox <gi...@apache.org>.
8431 commented on issue #11627:
URL: https://github.com/apache/dolphinscheduler/issues/11627#issuecomment-1237601362
please assigned me again .thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] [dolphinscheduler-common] When an application is deployed on the EC2 machine on AWS and S3 is operated, it will be rejected [dolphinscheduler]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #11627:
URL: https://github.com/apache/dolphinscheduler/issues/11627#issuecomment-1752205349
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] 8431 commented on issue #11627: [Bug] [dolphinscheduler-common] When an application is deployed on the EC2 machine on AWS and S3 is operated, it will be rejected
Posted by GitBox <gi...@apache.org>.
8431 commented on issue #11627:
URL: https://github.com/apache/dolphinscheduler/issues/11627#issuecomment-1237582854
Can I submit code? assign to me
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] EricGao888 commented on issue #11627: [Bug] [dolphinscheduler-common] When an application is deployed on the EC2 machine on AWS and S3 is operated, it will be rejected
Posted by GitBox <gi...@apache.org>.
EricGao888 commented on issue #11627:
URL: https://github.com/apache/dolphinscheduler/issues/11627#issuecomment-1290053789
@8431 Hi, may I ask whether you are still working on it? Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] [dolphinscheduler-common] When an application is deployed on the EC2 machine on AWS and S3 is operated, it will be rejected [dolphinscheduler]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #11627: [Bug] [dolphinscheduler-common] When an application is deployed on the EC2 machine on AWS and S3 is operated, it will be rejected
URL: https://github.com/apache/dolphinscheduler/issues/11627
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #11627: [Bug] [dolphinscheduler-common] When an application is deployed on the EC2 machine on AWS and S3 is operated, it will be rejected
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #11627:
URL: https://github.com/apache/dolphinscheduler/issues/11627#issuecomment-1281663156
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #11627: [Bug] [dolphinscheduler-common] When an application is deployed on the EC2 machine on AWS and S3 is operated, it will be rejected
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #11627:
URL: https://github.com/apache/dolphinscheduler/issues/11627#issuecomment-1225327316
Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
* In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
* If you haven't received a reply for a long time, you can [join our slack](https://s.apache.org/dolphinscheduler-slack) and send your question to channel `#troubleshooting`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] caishunfeng commented on issue #11627: [Bug] [dolphinscheduler-common] When an application is deployed on the EC2 machine on AWS and S3 is operated, it will be rejected
Posted by GitBox <gi...@apache.org>.
caishunfeng commented on issue #11627:
URL: https://github.com/apache/dolphinscheduler/issues/11627#issuecomment-1244926302
Hi @8431 I had assigned you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] [dolphinscheduler-common] When an application is deployed on the EC2 machine on AWS and S3 is operated, it will be rejected [dolphinscheduler]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #11627:
URL: https://github.com/apache/dolphinscheduler/issues/11627#issuecomment-1763551631
This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org